I've spent the last couple of days soaking up as much info as I can from the tapeheads site. Steep learning curve but it's fascinating stuff. Lots of disagreement too wrt the best way to calibrate, which is rather frustrating for a newbie seeking black and white answers, but I guess few answers are ever black and white!
Here are the basic steps for calibrating a machine, however you should allways follow the service manual for specific steps and levels associated with the actual machine :-
Before starting check the 1k output @ 0.775dBU from your signal generator with an AC Volt meter and cross ref on your Scope even an TRMS DVM will be accurate @ 400hz and maybe 1K
Input / Output levels
Using a signal generator with a 1K test tone @ (check your manual) 0.775 VRMS or 0dBu check the output of your signal gen with a calibrated meter then plug it in the input of the machine and set the machine to monitor input
Check the output of the machine so that it reads what the manual says and where it says to check it from, for this example lets say it should be exactly the same as the input which is correct for most machines, however it also depends if the machine has a CAL button setting?, if not then the level vol control for each channel needs to be set in a certain position normally 7 or 8
If the output level is not reading correct then check the manual for which pot to adjust for the output level
Now look at the VU meters and they should be reading 0VU, if not adjust the meters again check the manual for a adjustment pot.
OK now your machine is set for the correct input and output and your meters are reading 0VU.
Playback levels.
Now load your MRL Calibration test tape, I like to run this onto 10" spools to keep the tape tension correct
make sure your machine is in safe and set to monitor output with an analogue AC volt meter
Play the tape, 1k tone at 0dBu and adjust the playback level pot so the VU meters read 0VU , check your manual as some older decks use 400hz or 700hz as the ref playback level.
Now you can adjust the Azimuth of the playback head using the 1K & 10K tone, plug your oscilloscope into the output or monitor out for both channels, set the scope for AC and adjust so both sine waves overlay each other perfectly and take up approx 80% of the screen adjust the azymuth screw on head with a non magnetic screw driver or allen key and adjust for the maximum level and even output for both channels. start with the 1k tone then fine adjust with the 10K, when you get that correct switch the scope into XY mode and check you should have line which slopes upwards at 45 deg from bottom left to top right if its an o shape its 90 deg out of phase if its laying to the left its 180 deg out of phase see picture below.
Now you can adjust the Playback HF EQ, now on decks like my studer A810 or tascam br20 I can do that at 0dBU level and the tape will not saturate with modern tape
But on older machines like a B77 / A77 you will find that the 10K and 16K signals are at -10dB so the tape doesn't saturate and give you a false reading so check your reference tape.
play the 10K tone (-10dB) and set the AC volt meter to -10dB with the HF EQ trim pot The VU meter should also read -10VU
If the test tape has a 16K tone play that and it should read the same as 1K and 10K tweak the EQ and playback levels to get this as flat as possible.
If your machine has a LF adjustment pot play the 100hz tone and again set for the same level as 1K
Record levels
remove your test tape and store it in a safe place away from mains leads and any magnetic sources
Load the tape you wish to use for calibration lets say LPR35 or LPR90 and connect an AC Volt meter at the output
Connect your signal gen to both channels set the machine to record and monitor the
input and input a 1K signal @ 0.775 Vrms the meters should both read 0VU still
Switch to monitor
output and select Record&Play, the meter should read the same, if not adjust the record level pots.
Now check the Record head alignment Azimuth with the machine in monitor
output so you are listening to the playback head, now play a 10K tone @ -10dB and adjust the Record head azimuth so you get the maximum reading even on both channels and monitoring the sine wave on the scope at the output as you did for the playback head.
Now with the same 10K tone @ -10db hit play and record and we are going to adjust the record Bias level (
not the bias traps do not touch those pots)
On one channel at a time turn the bias pot down by turning anti clock wise the VU meter should start to rise peak then fall as you are still turning anti clockwise when it falls then start turning it clockwise until it peaks and note the peak point now you are going to overbias form that peak point by continuing to turn clockwise the meter should fall, keep turning until the you have overbiased by the amount stated by the tape manufacturer for that speed so for LPR35 that is 4.0dB from the peak point, this should give the lowest distortion for that tape at that speed.
Don’t worry if the bias peaks are not exactly the same for each channel that is normal
After adjusting the record bias and azimuth go back and check the 1K record playback as it might need a tweak
now with the same 10k tone @-10dB adjust the HF record EQ so the output levels read the same as the 1K level.
do the same with a range of HF levels I like to sweep from 1K to 20K and get the flattest frequency response.
For record bass you will probably find there is no record bass adjustment but if there is then record a 100hz tone adjust the LF “Playback” level until it is the same as 1K
Record some music and pour out a nice glass of red wine and listen to it
Hope this helps
Alan