First Round of Troubleshooting

You finally made it. After working 15 hour days for a week, the system is installed. The band arrives and soundcheck begins. You're excited because you installed an Aviom system, and setting up monitors should be a breeze (mainly because you don't have to do it!). You call for the kick drum, dial up the gain and stand back to wait while the musicians set the level on their personal mixers (which you've thoughtfully labeled with board tape). What you see, however, is confusion on the faces of your friendly musicians—more than usual.

Finally someone yells out, "Where's the kick? It's not on channel 1!" Quick, check your patch, "Uh, yeah it is!" "No, it's not. Oh, it's on channel 2." Call for the snare, which should be on channel 2. "That's on channel 3," is the response. You talk into the talkback, which should be on channel 14. "Talkback is on channel 11!" Now you're confused. You call someone else over to re-check the patch. Sure enough it's all showing correct, but nothing is where it's supposed to be.

That was our first Sunday. We ended up spending about 30 minutes try to track down the problem--and never did. We knew we were in deep weeds when we decided to reboot the board, and the Aviom channel assignments showed up different than they were before the power cycle, that is, a whole different wrong. Strangely enough, both the desk (a brand-new Yamaha 01V) and Studio Manager showed the patch as correct. Eventually the worship leader looked at me and said, "Sess, what do we do?" I made the decision to move ahead. We'd figure out where the stuff was on the fly, have them build their mixes and hope nothing changed during the services. It didn't. The day was saved--barely.

After tear down, Erik, my trusty FOH engineer extraordinaire, and I stayed around for an hour troubleshooting. We followed the same principles of troubleshooting I've written about before. We kept moving up the signal chain, removing components trying to figure out what was causing the problem. We swapped Aviom cards. No change. We bypassed the Aviom distro, going straight out of the card to a personal mixer. No change. And when I say "no change," what I mean is every time we powered off and back on, the Aviom channel configuration was different. Which is bad. In case you were wondering.

Someone who knows the board stopped by the tech booth to take a quick look. His pronouncement was that the output patch libraries must have been screwed up. The randomness of the problem made that seem unlikely, but we re-loaded the factory defaults anyway. We re-set the patch, and the randomness continued. We recalled all the factory defaults, still randomness. We gave up and went drinking.

Today, I tackled the problem anew. After a phone call to Yamaha tech support (which, by the way, was "closed until 1:30 PST for a company meeting." Seriously?), I was instructed on the fine art of re-initializing the board (hold down "Store" while powering up). No dice. On a whim, while I was talking to the woman at Yamaha, and getting ready to re-initialize again, I decided to pull the word clock and ADAT cables for our external Presonus Digimax 96 preamp.

Now, the reason we were sold this particular preamp is because it's supposed to sync to the 01V. And it does; if you don't mind a constant stream of digital noise coming through on all 8 channels if you try to clock at 48K. We don't, so we ended up using the pre as a word clock source. Oddly, this configuration produces no noise. It's not ideal, but it should work. Should being the operative word.

So after pulling the ADAT and word clock cables, I re-initialized and tried again. Hey, it works! I power cycled the board 4 times and the talkback mic stayed in the same Aviom channel every time. Victory! After we patted ourselves on the back, I hung up the phone with the friendly Yamaha lady and started re-connecting things. I plugged in the ADAT line, the word clock line and immediately was faced with a word clock error. I re-set the 01V to use word clock in, and again, on a whim, tested my talkback mic. It had moved to channel 16 of the Aviom. What the...

I power cycled the board. This time, talkback was on channel 13. I pulled the ADAT and word clock and it went right back to 14, where it belongs. Now, this is where it gets freaky (if that wasn't freaky enough). I decided to try synching word clock at 44.1K instead of 48K. Now, we've already determined that the Digimax won't properly clock to the 01V at 48, but would it clock properly at 44.1? Yes it does! And the Aviom channel assignments stay put. This is really weird because the Aviom operates at 48K, not 44.1K.

I even tried the Digimax as a clock source at 44.1K and everything worked fine. As soon as I switch to 48K, the channel assignments go haywire. So now, we have a few options. Send the Digimax back and get a different model (and perhaps another channel of wireless with the money we save); keep the Digimax and get an external clock source to see if it improves things; or run everything at 44.1K. The later option works, but it ticks me off that we have to run at 44.1 and upsample to 48 for the Aviom. Spending even more on an external clock source makes spending the extra money on the Digimax seem like a real waste.

We don't yet have a course of action, but at least we can get through this weekend while we figure one out. When we do, I'll let you know what it is, and how it works out. In the meantime, remember--when troubleshooting, isolate as much as you can right from the start to see if you can get something working. And remember Occam's Razor: Make as few assumptions as possible when trying to explain a phenomena. Everyone assumed it was a software issue; which really made no sense given the randomness of it. A bad patch, or a recall safe, or a bad output library would be wrong, but it would be the same wrong every time.

That's my lesson for the week.