Since Wednesday I have had an issue where none of my files were synchronizing from the primary server to any of the child servers.  I tried everything I knew to try under the sun.  I called tech support and we worked though everything under the sun again.  The issue was escalated up to professional services and we tried everything under the sun.  Nothing worked.

Now I should note in all the instances above each time the same basic things were tried, no one had any additional items to try just what I would call the standard things you do with Insight when it isn’t working as expected.  The biggest hint into the problem happened after the standard troubleshooting when client care and I moved two servers from my production FSU to my development FSU and we saw the DPUs synchronize.  We then moved them back to production and say them start to synchronize again.

Could it be all was solved?  Alas no, synchronization stopped mid way through.  In that process we had renamed the Synchronization.log, which had grown to over 6 gigs in just under four days.  Doing this caused the partial production synchronization to create a new smaller readable log file.  Upon investigating the synchronization.log everything seemed to be fine until it encounters one file and then we see nothing but errors.  What was that one file, a blank workspace file in a user workspace folder.  So even though this user doesn’t have permission to save anything to the server, the system saves a copy of the users local files to the server in the user directory and then synchronize that out to all servers.  A .vw with no name file was the cause of the whole system not being able to synchronize.

I couldn’t have found this out without having support on the phone with me every step of the way.  It was great to bounce ideas off of someone and get feedback and confirmation on troubleshooting steps.  While the troubleshooting took around 4 hours phone and screen sharing time, and additional hours of research.  It was very much a needle and the culprit was not something I thought could bring down an entire Insight system.

If you ever have any kind of synchronization issue and the sync log is too large to review use the windows Find command to filter things down a little bit.  Find is installed on every Windows system I would use the following command:

find /i “error” “d:\Visual Server\Trace\Synchronization.log” > error.txt

Then you can look through the error.txt file to see if any error might be causing the issue.