Sea Ray like many manufacturers did use balsa coring in hulls and decks, that is not necessarily cutting corners, as it is a viable method of construction and still used today.
If the core stays dry, its a light weigh, less expensive construction method. The key is to keep the core dry. That means rebedding any protrusions through that core material on occasion, and seeing as your boat is over 20 years old, it is way past due.
Its also possible that in 2004 there was little or no moisture, so its possible the dealer didn't lie. Did you get an independent survey at that time?
Now onto what to do today? Why did you get the survey, was it related to a sale of the boat or was this a requirement by your insurance company?
Not all surveys are the same quality. I would always get a second opinion especially if you are hearing this from a potential buyers survey. When I sold my last boat the first survey turned up "HUGE" amounts of moisture in the hull and deck, the largest amounts funnily enough were in areas with no core. My suspicion was the buyer was looking for an out and asked the surveyor to find something wrong fast so he could use that as an excuse, the surveyor was on site for less than 10 minutes.
Good luck, hopefully this was just a false alarm, but the reality is that you most likely have some moisture, I would venture a guess that more than half the boats over 10 years old have some moisture and 99% over 20 years old have some moisture.