Dataquest

Data Recovery You Can Trust

Category: In Depth

This is the section for our longform, long reads & detailed articles. Usually (but not strictly) on a single subject, and longer than 400 words.

  • OWC Mercury Elite Pro Qx2: Data Recovery

    OWC Mercury Elite Pro Qx2
    OWC Mercury Elite Pro Qx2

    This OWC external enclosure is a common sight on the desks of Mac users with big storage needs. It’s a pretty standard 4-bay box, styled somewhat like a cousin of a PowerMac G5 or 1st generation Mac Pro. Inside are the usual options of RAID 0 to RAID 5 with a few additions like JBOD & RAID 10 thrown in for good measure. There are a few variations of this device but the back panels commonly have USB, Firewire, and eSATA ports for direct connection to a PC or Mac. There is no ethernet port on these drives which makes the Qx2 a DAS (Direct Attached Storage) rather than NAS (Network Attached Storage).

    Aside from massive name, the OWC Mercury Elite Pro Qx2 also comes with a potentially huge amount of storage. Currently up to 32TB on the OWC store, but also available diskless or BYOD (Bring your own disks). With so much storage space, these drives often become the one and only repository for vast lumps of important data. The benefits of RAID give a false sense of security that the data is safe from drive failures. Unfortunately, there are a number of reasons why the RAID array alone will not protect from certain failures. Most of  these failures can be overcome by us in our workshop, but they are not one-button fixes. It is helpful to understand why a seemingly rock-solid platform can be even more risky than a simple external USB drive.

    Redundancy

    Under common settings, the Qx2 will use RAID 5 for the array. With four 2TB drives this gives you a 6TB volume on a Mac or ~5.5TB on a PC[1], and can cope with a single disk failure. There is a lot of debate about how good RAID 5 really is for such large drives[2]. In our example this means that if a single disk fails, it will need to be replaced, and then the new disk rebuilt with 2TB of data calculated from the other disks. This will take many hours, even under optimal conditions, but if anything goes wrong before it completes the array could stop showing up all together. At this stage, the data is probably recoverable but don’t panic. One wrong move and the data could be gone for good.

    If the data is crucial then get assistance from a RAID recovery service now and you should get back all or most of the data.

    If any disks are removed or replaced at this point the array could get reinitialised and either make the recovery more complicated or wipe the data completely.

    Other Failures

    Aside from all the problems with a RAID setup, the volume could also fail in the same ways that a standard hard drive can. There could be deleted files, a reformatted or corrupt partition, or even the RAID controller failure. RAID cannot protect against those types of failure at all.

    Recovery

    Our first step would be making read-only copies of each disk in the array. This protects against further disks failing, and also allows us to work from copies without risking the original disks. In fact, once the disks are copied, we put the originals to one side and don’t touch them again until all the data is recovered and supplied back to the user.

    Once we have our copies, they are loaded into our own hardware where we recreate the RAID in a virtual environment. Again, we don’t use the original hardware, as that may have been the root cause of the problem.

    When the virtual RAID has been loaded and all the data extracted, the files are supplied back on whatever alternative storage is suitable, (not the original device!) Once the data has been delivered to the user, and backups made, the old unit can then be destroyed, or returned and reused.

    Avoidance

    Anyone using RAID on a regular basis should know that RAID is not a replacement for backups. If anything, the increased number of disks makes failure more likely. This needs to be addressed by either making backups to another device, or an online service (preferably both). You ideally want backups that keep historic versions of the files, so that inadvertently deleting a file or changing a file by mistake will not also replace the backup version.

    If you are having problems with an OWC Mercury Elite Pro Qx2, give us a call or send a message via the form on this page. We give free advice and could help you avoid permanent data loss.

    1. Macs now use 1000 bytes for 1KB but PCs use 1024 bytes.

    2. Even RAID 6 does not solve the long time required to rebuild a disk, even though it allows for two disk failures.

  • Data Recovery from Fire Damaged Hard Drives

    Data Recovery from Fire Damaged Hard Drives
    Fire Damaged Hard Drives

    Following a fire at a science lab, five hard drives had sustained damage. Although hard drives often survive a fire, they can sometimes be damaged more by the water used to bring the blaze under control. Fortunately for the lab involved, somebody had managed to extract the hard disks from the scene and quickly bag them. It is critical to work fast with fire and water damage, before corrosion takes hold.

    The lab have been using our data recovery services since 2005 so the technical department knew exactly where to send the disks.

    Cleaning and Decontamination

    Data Recovery from Fire Damaged Hard Drives
    Data Recovery from Fire Damaged Hard Drives

    To maximise the chances of recovery, we have a strict procedure for fire and water damaged disks. First the outside covers of the drives are cleaned of any loose soot and all electronic components are labelled and removed. The hard drive carcases then get thoroughly cleaned and inspected for signs of water ingress or damage to the protective seals. If the damage has transferred inside the drive then it will be taken into our cleanroom for internal decontamination and cleaning. The electronic parts of the disks are dealt with separately. First they are dusted of loose debris, and then immersed in a chemical bath. This removes contaminants from any connectors or contact surfaces, and also helps remove anything that could cause the electronics to malfunction when powered on. The circuit boards are then dried and tested for faults before being reattached to the hard drives.

    Recovery

    Once cleaning and decontamination is complete, the drives are reassembled and attached to an imaging machine. The drives are copied as fast as possible, as they may have been exposed to temperatures outside of their specified design. This process means that each sector is only read a single time and then the disk is powered off and returned to storage. We are then free to work on the copies. It is part of our standard data recovery procedure, but all the more important in this case.

    Success

    Our strict and thorough process for fire and water damaged drives meant that we had a 100% success rate from these drives. Failure to follow any part of the process could have meant the difference between the data being recovered or not.

    Fire Damage Data Recovery

  • We Recovered a Western Digital Hard Drive That Others Couldn’t

    We Recovered a Western Digital Hard Drive That Others Couldn’t

    Case Study

    Drive: WD3200AAJS-40RYA0
    Problem: Clicking. Diagnosed by multiple third parties as unrecoverable.

    This hard drive has travelled the globe and had been looked at by various data recovery companies, eventually landing on the desk of our data recovery partner in Amsterdam. Our partner then asked us if we were up for a challenge and explained to us that the last data recovery company to look at the drive in Portugal had given up with no success.

    Third Party Feedback

    We got in contact with the company in Portugal and they explained to us that they had tried mechanical and firmware repair, but both did not overcome the problem.

    Booking In

    Once the drive was booked into our process, it was clear that the warranty seals had been removed and the top cover had been previously opened. Therefore we needed to take the hard drive straight into our clean room to check for any media damage and also to confirm that the mechanics of the drive had been rebuilt correctly.

    Findings

    We found no cause for concern in the clean room examination. The drive appeared to be clean and was correctly rebuilt. We replaced the internal heads with a new set as a matter of course and rebuilt the drive and then moved to the next stage of our process. The drive was then powered up on our proprietary equipment and an extensive firmware check and fix process was carried out. Perseverance was the key to overcoming the firmware corruption. It was apparent that a lot of the original firmware had been totally overwritten and a lot of painstaking work was done over the next few weeks to rebuild the firmware to eventually allow the drive to come ready.

    Imaging

    Once it was established that we could access the data area of the drive, it was put into a block level head map imaging process. This allowed us to control the drive and to further stabilise it by imaging each disk surface individually. Once this process was complete a file extraction was carried out on the image to reveal the original folder and file structure.

    Result

    The image process was a complete success and resulted in over 190GB of data recovered. A  folder and file structure list was sent to the customer, who acknowledged saying that all the data had been recovered.

    Once again we were able to test ourselves against our competition, all be it in Portugal.  In this case the customer was fortunate to seek advice from our data recovery partner in Amsterdam who recommended us. We can’t help but wonder how many people are told that their data is lost for ever, without the thought of getting a second opinion.

    If you need data recovery from a similar hard drive, you may be interested in our Western Digital Data Recovery Services.

  • How To Ruin Your Chances Of Data Recovery

    How To Ruin Your Chances Of Data Recovery

    Maybe the title is a bit harsh, but in the past week I have seen an unusually high number of hard drives ruined by avoidable problems. There is nothing more frustrating than knowing the data would have been recoverable if  the hard drive hadn’t been tampered with first.

    Manufacturing

    Hard drives are manufactured in a controlled environment. Staff wearing white overalls,  gloves, and masks, control machines which are carefully organised to prevent contamination getting inside the hard drive.

    Clean Room

    To prevent contamination when repairing the inside of a hard drive, it is necessary to use a cleanroom. This is a specially designed system that filters the air and keeps airborne particles to a minimum. A cleanroom is the only safe way to open a hard drive. We have one of these, which allows us to carry out the most intricate repairs without any risk of particle damage.

    Damage

    Before opening a hard drive for internal rework, it is crucial to confirm that there is an internal mechanical problem in the first place. For instance, our data recovery process has numerous tests we can carry out before we confirm that the problem is under the hood. Only then will we open the top cover and check for damage inside. Unlike CDs or DVDs, the internal disks of a hard drive are not designed to come into contact with normal air. Even a fingerprint could mean the difference between a successful or unsuccessful recovery. If you slip with a screwdriver then forget it.

    The numbers

    On a percentage basis of all the failed drives we receive (hundreds per year), we only need to go to the cleanroom with around 10%. There are likely to be far less intrusive ways into the data without ever taking off the top cover.

    Even if…

    I’ve lost count of the number of times I’ve been told, “It’s an internal fault. The heads are clicking.” Although some drives will click as a symptom of failed heads, there are so many other reasons to cause a drive to click that it is not a reliable way to diagnose problems.

    Reasons a hard drive will click

    • Electronic fault
    • Firmware fault
    • Bad sectors
    •  Weak (but not completely failed) heads
    • Problems reading partition info

    All of the problems above can be overcome without ever unscrewing the top cover. In fact, removing the top cover will only introduce more doubt to the diagnosis. If the top cover is removed outside of a cleanroom, then not only do the above problems need to be solved, but also new problems of contamination and possibly damage by tools.

    (Don’t) Get the discs out

    Another common wrong diagnosis is to take the disc pack from one drive and place it into a donor drive. Although this is a correct course of action for some drives, there are some serious implications. First, when the disc pack is built it is clamped together onto the spindle motor. This alignment is so crucial that if you remove one disc at a time, you will never be able to regain this alignment. This was true 20 years ago when the magnetic data was nowhere near as densely packed. Rotate the discs a fraction of a millimetre and you can wave goodbye to all the files.

    It is also the case that most of the hard drive firmware is now stored on the discs, so moving them to a new donor will not help if the fault is firmware based.

    Advice

    If you are even considering a destructive course of action, at least get some professional advice first. You don’t need to follow the advice, but at least it gives somebody (maybe me) the option to give a warning. You don’t want to find out later that the data would have been simple to recover, if only…

  • 2TB Western Digital 2nd Opinion

    2TB Western Digital 2nd Opinion

    Case Study

    Drive: WD20EARS-00MVWB0
    Problem: Clicking. Previously diagnosed by a third party as unrecoverable due to media damage.

    We are always keen to test our services against our competitors, just to make sure we’re still up there with the best of them. When we heard from one of our service partners (ABC Rawpaw) that one of the top ranking data recovery companies on Google had given up on a customer’s drive, we were curious to take a look. It’s also worth mentioning that the client had paid £234 to the other company, despite them recovering no data.

    Third party report

    We received a copy of the original diagnosis report with the hard drive. The report mentioned incorrect head alignment and also damage to the head assembly preventing the drive from functioning correctly. It also stated that they had subsequently replaced the old heads with new ones but due to suspected disc damage, recovery was unsuccessful. A plausible enough report, but was it accurate?

    Booking in

    The drive was booked into our process. We noted that the warranty seals had been removed and the top cover showed signs of being previously opened. Whenever we receive a drive in this state, we always want to make sure the drive has been rebuilt correctly. We need to do this in our cleanroom to prevent any contamination getting in from the air. Our office is clean, but it’s no place to open a hard drive.

    Findings

    The inside of the drive was examined in our cleanroom and found to be spotlessly clean. There were no signs of disc damage or particles, but we did notice that the screws were not factory-tight, suggesting previous work had taken place. Happy that there was nothing untoward inside the disk, it was rebuilt for further diagnosis. The drive was powered on and initially failed to reach a ready state. This is common for Western Digital drives as they often have firmware corruption. We used our proprietary firmware tools (and John’s keen knowledge) to repair some of the firmware area of the drive which then allowed the drive to reach a ready state and allow access by our imaging tools.

    Imaging

    We did find that one of the heads was not performing within spec, but we were able to work around this, again using specialist tools. After imaging the majority of the drive we replaced the heads to allow better access to the missing areas. This helped improve the amount of successfully recovered data.

    Result

    We ended up with over 800GB of data recovered from the drive in good condition, even though one of the biggest companies (according to Google) was unable to get anything! This is not the first time we’ve recovered data where others have failed, but we always wonder how many people give up on ever seeing their data again, even though it may be recoverable with the right knowledge.

    In the end we had an extremely happy customer (and service partner), and were able to test ourselves against one of the biggest data recovery firms in the UK. A great result for us that shows the importance of getting a second opinion.

  • What To Do When You Run Out Of Space

    It’s a common problem that as we generate more data each year we start running out of space to put it. This is now even more of an issue in the smartphone market, where built-in cameras are generating increasingly large photos and videos, without providing much in the way of additional storage. The most common iPhones are still 16 & 32GB but the photos they now produce can be megabytes in size, with videos easily reaching 1GB.

    What To Do When You Run Out Of Space
    What To Do When You Run Out Of Space

    It’s tempting to take that data and put it somewhere else, so either a laptop or external hard drive. Then once you’ve copied it all you delete it from the phone and gain back all that space. Problem solved.

    Not So Fast…
    If that copy on your laptop is now the only copy, then you could be one spilt coffee from disaster. If the laptop goes up in smoke, gets stolen, dropped or any of the myriad other ways of failing then it’s bye bye data.

    The Fix

    The key to making backups is redundancy. The key to making backups is redundancy. The key to making backups is redundancy.

    You need to make extra copies of your data to different types of storage. This could be an external hard drive, NAS, USB Pen, SD card, anything. But don’t just pick one of those. Make a few backups. Put one in a locked safe somewhere. Send photos off to the cloud. Store a copy of your music at your nan’s house. If any of those copies gets lost or broken you can just replace it with another copy.

    So let’s run through an example. All those photos on your iPhone have filled it up. Here’s what I would do:

    1. Copy the photos to my computer. Check them.
    2. Backup the computer as usual. (You’re already doing that, right?)
    3. Make another backup, or copy the photos to an online storage service like Dropbox.
    4. Now it is safe to delete the photos from the iPhone and revel in all that fresh space.

    Note: Deleted photo recovery is virtually impossible for all modern iPhone versions due to encryption. 

    Here’s another example for when your computer runs out of space instead:

    1. Is it possible to upgrade the internal storage? If it is then you should do that.
    2. If this is not possible, or too expensive then you will have to get creative. It will be more fiddly but copy all data to two external hard drives.
    3. You always want to avoid just leaving your data in one place. All electronic devices can (and will) fail, and they have a terrible habit of doing so at the worst possible moment.

    So, just remember that no single copy of your files are safe. Making extra copies is cheaper and easier than waiting until something fails.

    Thanks to Alexander Armstrong for inspiring this post.

  • SSD Data Recovery

    SSD Data Recovery
    SSD Data Recovery

    SSDs (Solid State Drives) may one day become the standard form of storage in computers. Apple laptops are already heading that way. There are certainly many advantages when comparing SSDs to HDDs (Hard Disk Drives), however they do bring their own problems, which are often not well reported. We don’t care how good SSDs can be. We care about how they fail. It’s common to hear things like: “I’m replacing my hard drive with an SSD so I won’t have to worry about it crashing again.” While this is technically true – there are no moving parts to crash – there are plenty of other ways an SSD can fail. Whether it’s technically crashed or not doesn’t matter at all when you can’t access your files. It’s a shame but an SSD does not get you out of the boring task of running regular backups.

    There are some pros and cons which specifically affect data recovery from SSDs. I haven’t listed things like battery life or read / write speed as they are not relevant when it comes to recovering data from them.

    SSD Data Recovery Pros:

    • Shock resistance. No moving parts to crash.
    • Just as susceptible to filesystem issues, deletion, reformatting, bad sectors etc which can be recovered using existing equipment.

    SSD Cons:

    • False sense of security. The word reliable comes up a lot in SSD marketing with phrases like “More reliable, faster, and more durable than traditional magnetic hard drives.” Maybe research exists that shows SSDs are less prone to failure but it doesn’t seem to be the case at the moment. Anything that holds your valuable data runs the risk of getting drenched, getting stolen, getting lost, and that’s before we even take general failures into account.
    • Susceptible to electronic failure, Maybe more so than a hard drive as the storage and electronics are combined in SSDs. Some of the most common hard drive failures are caused by errors in the firmware which controls the performance of the drive. SSDs have very complex firmware, which opens the possibility of firmware corruption. In most cases firmware corruption will block access to your data.
    • Encryption. Most modern SSDs encrypt the data at a hardware level, which makes it impossible to remove data chips and extract data from them externally (you can do it, but the data is encrypted). The keys to the encryption are often stored within the controller chip, so if that fails, you could be locked out of your data for good. Modern encryption works well. You can’t get round it.
    • Wear-levelling algorithms. Which move the data around the SSDs to improve performance, can make recovery difficult as these algorithms would need to be taken into account when accessing a failed SSD. They don’t store data in logical order like hard drives do.

  • How Hard Drives Store Data Across Multiple Heads

    In most computers, when you save files they get stored on a hard drive. Although you wouldn’t know it, the drive does not store your files in a straightforward way. The data is written magnetically by a fixed comb of heads stacked above one another. These heads pass between several magnetic discs, writing data as they go. In most cases, instead of storing files on one whole disk they are split up and spread across the disks. This means that when we carry out data recovery we usually need all of the disc surfaces in good condition to get the data back.

    How Hard Drives Store Data Across Multiple Heads
    How Hard Drives Store Data Across Multiple Heads

    When required we can use a process to take the data from the drive by one disc surface at a time. This can allow us to avoid a failing head until we have the rest of the data extracted. When we have extracted all of the data the parts are combined to allow access the files. In some cases this is the only way to get the data back.

    Hard drives do not allow access to individual disks during normal operations so we need to use specialist hardware and software.

    We commonly need to access individual heads on Hitachi drives, due to degraded magnetic discs. Also if a drive is dropped when in-use, it will often damage at least one head.

  • Data Recovery Success Rates October 2012

    October was a pretty busy month for us, so I thought it would be a good chance to check on our success rate. As you can see from the graphic below, we have a great success rate of at least 69%. We always keep an eye on our success rate, to make sure we are still recovering as many drives as possible. Our success rate is often higher than 69% but we did get a few non-recoverable drives which had suffered physical media damage. For an example of why those are unrecoverable, have a look at a photo of a head crash. (Tip: Those dark circular lines are not meant to be there!)

    Data Recovery Success Rate October 2012
    Data Recovery Success Rate October 2012

    Of those successful jobs, a whopping 90% of them were recovered without even needing to repair them in our cleanroom. This is interesting as cleanroom facilities are often advertised as one of the most important factors when choosing a data recovery company. Not to undermine the need for cleanroom facilities, but they are not required for most hard drives.

    Data Recovery Success Rate October 2012 Split
    Data Recovery Success Rate October 2012 Split

    In the graphic above we have classified non-cleanroom jobs as external, and cleanroom jobs as internal.

     

  • How a Data Recovery Engineer Sees RAID

    As somebody recovering data from RAID arrays, my view on them is a little different to the norm. In most cases I would say avoid RAID wherever possible. Simplicity is key.

    Below are my answers to some real questions I have received from clients about RAIDs.

    Why did this RAID disk fail?

    Hard drive failure is not unusual and is often not avoidable. The truth is that all hard drives fail eventually, whether they are used in a RAID or not. Even though a RAID system can provide some fault tolerance from physical drive failure, they do have limits. A RAID5 on three disks for example can only handle a single drive failure at any one time. It is common for a second disk to fail whilst the other disk is being replaced. This is when RAID recovery is required; to first access the failed drives, and then rebuild the RAID. The best protection against RAID failure is to make backups. Backups in as many formats, in as many different physical locations as possible.

    Why did the server fail so badly? Isn’t RAID meant to prevent this?

    A 3-disk RAID5 can only cope with one bad disk. This doesn’t help when two drives fail at the same time. Although a RAID array can provide some leeway when it comes to disk failures, it doesn’t always help when you have multiple failures in quick succession. Adding more disks to the RAID can provide more redundancy, however this costs more money, and also adds complexity when things go wrong. Also you could be in a similar position if three disks happen to fail next time. A live system could fail at any time so prepare for the worst. Backups are cheap, and take a relatively short amount of time. RAID recovery can be expensive and cause unnecessary downtime.

    Why couldn’t our IT support recover this?

    We are a specialist data recovery company, with access to tools and resources which are not available to IT Support staff. We have spent the last fifteen years perfecting the process of extracting data from failed & failing hard drives and RAID arrays. For the best chance of recovery, we like to get the drives as soon after failure as possible. If more work gets carried out on the drives, things can be made much worse.

    How can we avoid this happening again in the future?

    To avoid similar problems in the future, the best way forward is some form of regular backup. The backups should be verified and then tested / restored as often as possible. This is where disaster recovery comes in, which can involve simulating certain types of failure and making sure you can get up and running again from your backups. At the very least, it wouldn’t hurt to put the really crucial business files onto an external hard drive every few weeks and store it in your company safe. It’s low-tech but at least you could plug it in to any PC and access the important business data if required as a last resort.

    I’m not against RAIDs. They do have their place, but cannot be relied upon as a replacement for regular backups.

    We have more articles about RAID here.