Naked Science Forum
Non Life Sciences => Technology => Topic started by: guest39538 on 18/03/2016 10:16:27
-
Hello, if I was recording on a camcorder to a hard disk, would the information ''fill'' the hard drive at a constant rate?
-
Speed is very specific to the particular drive. While the platters inside are rotating at a constant angular velocity, the modern hard drive tries to record data at a constant data density per unit length of head travel along the track, so the data rate varies as the head moves from inside of the disc, where the lowest block numbers are, to the higher numbered blocks that are at the edge. The data rate further is going to be varied by the encoding applied to the data so it can be recorded reliably, with forward error correction and spectrum spreading data added, along with the encryption if used on the drive, so the rate will be varying around a small value that is compensated for by the buffer built into the drive, typically something like 64k, 128k or 256k, depending on the drive type and desired application.
Then you get variations caused by the drive having to move the heads, and after moving it has to check it is on the correct track, by reading some sector data, then writing when the right sector is below the head, and then reading the next sector data before the write. Also complicating things id the drive remapping bad sectors, so that you can have the next logical block to be read or written not being actually in the next physical sector, but it has been relocated to a spare block ( interspersed through the drive surface during manufacture and hidden, like so many of the internal operations, from the outside, so the drive appears as a perfect drive while in reality it is very unreliable, relying heavily on error detection and correction to get the data back and show it to the outside) so there has to be a head movement and then a few cycles to get the correct track. Tracks are so close together that the only way to get the correct one is to go to the approximate position and then move slowly while reading to get the correct track and then wait for the right block to go under the head.
Reading can be worse, as the drive often has to use error correction to get the data back despite noise, or do multiple reads to reconstruct the data from best guesses from the reads. Too many and it ( secret sauce again) will decide to reallocate the data on the block to a spare track when idle, and mark the block internally as relocated and not usable. Thus a drive which appears as perfect can go from working to unreadable very fast as the spares are used up, and the drive no longer can swap out growing defects.
But to the original question, so long as the data is coming in at a lower rate than the worst write ability the drive can keep up and write it, if it comes in the drive will buffer to the point where the buffer is full, then simply discard some data ( mostly the last lot) and return an error code of it not being able to write the data. Reading the data rate will be set by the drive, requesting faster will simply result in the drive returning as not ready until the buffer is filled with data, and the read will stall.
-
Excellent post. My own knowledge of data storage is 70s vintage, onto tape. It was interesting to read of some of the complexities of almost current technology.
-
Yes, excellent post; thanks.
-
Speed is very specific to the particular drive. While the platters inside are rotating at a constant angular velocity, the modern hard drive tries to record data at a constant data density per unit length of head travel along the track, so the data rate varies as the head moves from inside of the disc, where the lowest block numbers are, to the higher numbered blocks that are at the edge. The data rate further is going to be varied by the encoding applied to the data so it can be recorded reliably, with forward error correction and spectrum spreading data added, along with the encryption if used on the drive, so the rate will be varying around a small value that is compensated for by the buffer built into the drive, typically something like 64k, 128k or 256k, depending on the drive type and desired application.
Then you get variations caused by the drive having to move the heads, and after moving it has to check it is on the correct track, by reading some sector data, then writing when the right sector is below the head, and then reading the next sector data before the write. Also complicating things id the drive remapping bad sectors, so that you can have the next logical block to be read or written not being actually in the next physical sector, but it has been relocated to a spare block ( interspersed through the drive surface during manufacture and hidden, like so many of the internal operations, from the outside, so the drive appears as a perfect drive while in reality it is very unreliable, relying heavily on error detection and correction to get the data back and show it to the outside) so there has to be a head movement and then a few cycles to get the correct track. Tracks are so close together that the only way to get the correct one is to go to the approximate position and then move slowly while reading to get the correct track and then wait for the right block to go under the head.
Reading can be worse, as the drive often has to use error correction to get the data back despite noise, or do multiple reads to reconstruct the data from best guesses from the reads. Too many and it ( secret sauce again) will decide to reallocate the data on the block to a spare track when idle, and mark the block internally as relocated and not usable. Thus a drive which appears as perfect can go from working to unreadable very fast as the spares are used up, and the drive no longer can swap out growing defects.
But to the original question, so long as the data is coming in at a lower rate than the worst write ability the drive can keep up and write it, if it comes in the drive will buffer to the point where the buffer is full, then simply discard some data ( mostly the last lot) and return an error code of it not being able to write the data. Reading the data rate will be set by the drive, requesting faster will simply result in the drive returning as not ready until the buffer is filled with data, and the read will stall.
Wow , I thought I knew a bit about computers etc, I am truly humbled . I would pretty much consider you an expert in your knowledge of hard drives and would appreciate your opinion and answers.
1. The hard drive when ''blank'' contains only 0's?
2. Two identical camcorders will record at an equal rate?
3. Two synchronous recordings on the two individual hard drives will show the same amount of space used on the hard drives?
4. Both recordings will show the same amount of time recorded?
-
1. The hard drive when ''blank'' contains only 0's?
No. Part of the manufacturing process is to "format" the disk into tracks and sectors, identifying bad sectors.
The spectrum spreading mentioned above uses a pseudo-random bit sequence, so that the data on the disk appears to be random.
2. Two identical camcorders will record at an equal rate?
They design the camcorders to produce video at a lower rate than the drive can store data, to minimize data loss.
If there is a "glitch" due to hitting a bad sector or two, the camcorder has a large buffer memory (outside the disk itself), where the data can sit until the disk drive "catches up".
Some video compression algorithms can encode at a lower resolution(ie recording the data at a lower rate) if this buffer starts to fill up, further hiding the recording pause.
3. Two synchronous recordings on the two individual hard drives will show the same amount of space used on the hard drives?
Depending on how they manage bad sectors, software tools may be able to reveal that new disk drives from the factory have different amounts of available storage.
If variable-rate video coding is used, the size of the video file could vary between two recordings of the same scene.
It is very hard to get two camcorders to record the same scene! Half-silvered mirrors, anyone?
4. Both recordings will show the same amount of time recorded?
Yes. Unless data is actually dropped, the recorded time should be identical.
-
would the information ''fill'' the hard drive at a constant rate?
No.
All commercial video coding (turning photons into bits) aims for a certain video quality, including vertical/horizontal resolution (eg HD, SD, etc), bit depth (eg 8 or 16 bits) and frame rate (50 fields/second, 120 frames/second). They are also constrained by the data rate they can sustain to a hard disk, a digital TV broadcast, or an internet connection.
To minimize the data rate and provide uniform quality, they encode fast-moving parts of the image at a high rate, and slower-moving parts of the image at a low rate. In theory, if the image is not moving at all, they would not record any data, but for practical reasons, that doesn't quite happen.
So if your image as a whole has a lot of motion (eg trackside at a Formula 1 race, following the passing cars down the straight), it will record onto the disk at a high average data rate. If your image has no motion (eg watching a condor's egg in a nest), it will record a very low data rate - random changes in pixel intensity and background noises in the sound track (plus occasional snapshots of the entire scene).
-
4. Both recordings will show the same amount of time recorded?
Yes. Unless data is actually dropped, the recorded time should be identical.
Thank you , I got agreement on the important answer from you. So just to confirm, if I was on planet X that was half the mass of the earth, and you was on earth, and we started to record synchronised , at the end of recording we would both have to agree we have just recorded the exact same amount of time as each other?
-
Just another caveat about the drive, as it comes from the factory, is actually blank. The one platter has been taken and had the head assembly ( the slots in the top covered by an adhesive cover and the label cover the holes) precisely aligned while a so called servo track is written on it. Then this servo track is used during test to write the rest of the tracks as the drive is used to scan itself. The drive itself uses a special firmware loaded in the factory to do this work, then the drive has the latest firmware version loaded as a final step before it is packed, the special firmware generates the first bad block tables, stored on the drive platter itself. The final firmware ( loaded by a small 6502 clone inside the drive that handles the initial startup of the drive by loading all the registers of the main processors and loading the software they need into them) is only run the first time when you power up the drive. This then will do a read after write on each write ( what it has to do in all cases anyway) to actually verify the data has been recorded, using a separate read head on the surface a little downstream of the write head.
The attached is a drive I scrapped this last week, where the one head separated from the head carrier ( never found it, just little bits of silicon dust) in operation, and disintegrated the drive surface on both sides. Lucky I had a backup on a drive that in turn also turned out to have issues, though I was able to recover the data I needed using a Puppy install on a DVD. Windows would not read the drive, but Linux had no issues, and SMARTMON stopped a little way in with unreadable sectors.
-
Just another caveat about the drive, as it comes from the factory, is actually blank. The one platter has been taken and had the head assembly ( the slots in the top covered by an adhesive cover and the label cover the holes) precisely aligned while a so called servo track is written on it. Then this servo track is used during test to write the rest of the tracks as the drive is used to scan itself. The drive itself uses a special firmware loaded in the factory to do this work, then the drive has the latest firmware version loaded as a final step before it is packed, the special firmware generates the first bad block tables, stored on the drive platter itself. The final firmware ( loaded by a small 6502 clone inside the drive that handles the initial startup of the drive by loading all the registers of the main processors and loading the software they need into them) is only run the first time when you power up the drive. This then will do a read after write on each write ( what it has to do in all cases anyway) to actually verify the data has been recorded, using a separate read head on the surface a little downstream of the write head.
The attached is a drive I scrapped this last week, where the one head separated from the head carrier ( never found it, just little bits of silicon dust) in operation, and disintegrated the drive surface on both sides. Lucky I had a backup on a drive that in turn also turned out to have issues, though I was able to recover the data I needed using a Puppy install on a DVD. Windows would not read the drive, but Linux had no issues, and SMARTMON stopped a little way in with unreadable sectors.
You really do know your stuff, so now if we consider a SATA hard drive and an attached SATA cable from the motherboard,
What substance travels down the cable to the HD that contains the information?
and by what method does the HD store the information? (magnetically I believe)
-
Data is sent down the cable as serial data, basically the SATA cable is an extension of a PCI slot, but slightly different data wise, but the principle is that data is sent as voltage level differences down 2 wires, and the direction is controlled by the motherboard controller which sends control commands and data to the drive, then reads a set of status registers and collects the data. Differential signals as they are less sensitive to noise on the lines, and with a little error detection built in, so the controller can detect single bit errors and ask the drive to send it again. If you start getting SATA cable errors the fault is the cable, or the motherboard port, almost never the drive.
As to how the drive stores the data it is as magnetic field changes along the track. The actual direction is not important, the data is encoded as variations in the field in time as the head flys over the surface, not quite touching but floating on a thin film of air ( or on newer drives Helium) so it does not touch the surface. The data is arranged in the encoding process ( the drive does not care about the data but just wants to store it) first with encryption if turned on ( newer drives that support hardware encryption on the drive so the data stored there is hard to recover if the user and the bios does not provide the drive with the correct key as the first transaction after power on so it can decode correctly) then it goes through a process of forward error correction, which adds extra data so that the limitations of the record and read system are not exceeded. Things like every run of eg 3 1 bits will have a 0 added, and a run of 5 0 bits will have a 1 added to the now serial stream. Then this stream goes through another encoder so the bits are scrambled up a little so that the write will succeed, and goes through another buffer where it will wait till it is written. then the drive goes to the rough area of the block, and looks in the servo data for the right track, then looks on the right disk surface ( there will always be more than 1 surface, typically 2 per platter but with the caveat that if there are never more than 4 platters and 7 heads as this then gets too thick for the casing) and reads the data on the disk looking for the right sector code. When this is read the write head immediately starts to write the block of data to the disk, starting in a small space after the sector number with a preamble ( so the read electronics can recover a clock to recover the data), then the data block, a CRC check for detecting a single bit error so the drive can either re read the sector next time it comes around, or use data correction to get the single or small number of errors back, and then a lead out block to show end of data. Then a small gap before the next sector number. Rinse and repeat till all data is written.
Reading is done nowdays with a GMR head, which is a quantum tunelling device, not a coil in a magnetic material, which is too slow. Read data rates are in the gigabits per second range, so the read electronics are on the actual head itself.
https://en.wikipedia.org/wiki/Giant_magnetoresistance
https://en.wikipedia.org/wiki/Hard_disk_drives
The Wiki articles are a good starting point if you want more.
-
Thank you , I got agreement on the important answer from you.
The original question was posed as the same model camcorder on Earth simultaneously viewing the same scene on Earth.
The answer was given in the same context.
So just to confirm, if I was on planet X that was half the mass of the earth, and you was on earth, and we started to record synchronised , at the end of recording we would both have to agree we have just recorded the exact same amount of time as each other?
Now a new question:What if the same model camcorder is now on two different planets of different mass (presumably orbiting different stars, possibly of different mass?), viewing different scenes? (The meaning of "simultaneous" is now a bit wobbly, since they aren't in the same laboratory).
Answer to the new question: The recorded times would be different, but by an amount that is too small to measure. The crystal clock in a camcorder has an accuracy measured in parts per million. To detect any difference between the flow of time in a gravitational field as weak as Earth's requires clocks accurate to parts per billion or better.
-
Answer to the new question: The recorded times would be different, but by an amount that is too small to measure.
Interesting Evan, I would of thought that the data recorded on the HD was an exact equal amount because the light being detected is constant?
So are you saying that in dual playback one will run slower than the other one slightly?
-
are you saying that in dual playback one will run slower than the other one slightly?
Yes, but you won't be able to tell the difference using this experimental setup.
The playback speed is controlled by a quartz crystal (the same one used for recording, with accuracy measured in parts per million); this measurement error totally obscures any differences due to relativistic effects.
Quartz crystals also suffer from frequency drift with temperature and "aging" (the frequency drifts over time). Relativistic effects are too subtle to measure with such crude tools; you really need to use atomic clocks.
See: http://en.wikipedia.org/wiki/Crystal_oscillator#Stability_and_aging
-
Is it possible that the latest Helium filled HD Drives will be considered sufficient reliable to use in space vehicles, I understand that tape recorders have been superseded by SSD,s
-
Helium filled drives will never be used in spacecraft ( neither will normal ones either, they are both too delicate to survive launch and do not work in a vacuum), as there is almost always going to be a modern solid state drive there, or in older craft a tape drive using metal film tape on reels.
The solid state drives are used as redundant pairs, with each part of the pair also having a massive amount of error detection and correction on them to both detect errors and correct them in use. As well the common consumer hard drive imparts a gyroscopic motion to the space craft, which means alignment of the craft will drift slightly, while a non moving drive does not have this effect. Unless you seal the drive in a very good hermetic container ( and this has to be good, with a loss of less than 0.1% of the fill gas per year from any reason) the bearings will seize up from outgassing of the lubricant and the heads will not be able to fly just above the disk surface on a cushion of gas.
While hard drives are used on the ISS in laptops, they typically are used only in a shirt sleeve environment, and are also sent up as cargo on crewed missions as well, with lower launch G loads and in a vibration absorbing case. Also there are redundant laptops, as they do fail up there regularly, and as it is often too difficult to change out the hard drive the whole laptop is often placed in the trash and burnt up in a garbage dump. They have been replacing the laptop drives with SSD units for a while now, just to get the speed increase and improved reliability from vibration.