We see various types of camera media come into us for data recovery, with surprisingly varied file formats. Many camera manufacturers use their own raw format, alongside various JPG options.
This raw format is sometimes known as a digital negative, containing (mostly) untouched data straight from the camera. These raw format images can be ten times the size of JPG images.
The benefits of raw files are to allow post-processing without the loss of quality from JPG files. Settings like sharpness, saturation and white balance can be changed at a later date using photographic software. Below are a few of the different raw file types in use.
The following is a description about some RAW formats:
CRW – Canon Digital Camera Raw Image Format. Raw image format for some Canon digital cameras. Raw images are basically the data as it comes directly from the CCD detector in the camera. Raw files can also contain text information about the picture and conditions in the camera when the picture was taken.
CR2 – Canon Digital Camera Raw Image Format version 2.0. Raw files can also contain text information about the picture and conditions in the camera when the picture was taken. These images are based on the TIFF image standard.
NEF – Nikon Digital SLR Camera Raw Image File. Raw image format for some Nikon digital cameras.
RAF – Fuji CCD-RAW Graphic File. Exif (Exchangeable Image File) information is within the file along with the image.
X3F – Sigma Camera RAW Picture File. Use the SIGMA Photo Pro software provided with the camera to download and manipulate the photos. The Foveon X3 direct image sensor captures all three colors at every pixel location and requires special software to manipulate the RAW files.
BAY – Kodak/Roper Bayer Picture Sequence. A specific Kodak picture format used by some high speed video cameras such as Kodak HRC-1000.
ORF – Descent 3 Outrage Room Format.
MRW – Minolta Diamage Raw Image File. Raw image format for some Minolta digital cameras.
RAW – Image Alchemy HSI Temporary Raw Bitmap
SRF – Sony DSC-F828 Raw Image File. CCD-Sensor RAW Data File from Sony DSC-F828 8 megapixel digital camera.
We have recently recovered a RAID 5 array which consisted of three of these ST373454LC SCSI hard drives. These are solid, weighty drives, which don’t give off a great deal of vibration, despite spinning at 15,000 rpm; 3 times faster than most laptop hard drives!
Upon opening one of the drives for cleanroom rework we discovered why these drives spin so quietly. In the picture below you can see that although the drives are standard 3.5″ form factor, they actually have 2.5″ disk platters. These smaller disks create less drag, and therefore can spin faster without stability problems.
These drives are not alone in mixing up the form factors. The popular WD Raptor drives also use a similar design.
Of course the biggest downside to using smaller disks is the lower storage capacity. Typically SCSI hard drives are much lower capacity than their SATA counterparts, so this trade-off is acceptable for the speed and reliability increases. The relatively low capacity is further mitigated when the drives are used in RAID arrays.
This is a video of a functional Toshiba laptop drive. Using a macro lens, we are able to see what happens when you read or write data to or from a hard drive. You can also see how the hard drive parks the heads off the disks when the power is shut off. This is to protect the hard drive from shocks during transportation.
It’s my first one of these videos so you can see my reflection in a couple of places, when I lean over to send different commands to the drive.
We have just received an 8GB USB pen with a broken connector. The customer started phoning around and was shocked by the costs he was given. He then called us and was pleased to hear that we only charged £90.00 if the rework was successful. This rework includes dismantling and repair of the USB Pen even when it appeared to be beyond repair.
Bang Goes The Theory – Series 6 Episode 3 – March 26th
I love Bang Goes The Theory. I loved the alcohol powered motorbikes last week and find it a good doorway into ideas, which are presented in a fun and interesting way. I was extra excited when I started watching episode 3, and relised they would be featuring data recovery. A perfect opportunity to dispel some common myths, and dish out a bit of advice in the process.
The data recovery guy Rob, made a good analogy when he described deleting data as ripping out a page from the table of contents. That is pretty much how it works, and really simple to understand.
Data Recovery Experts
Yes they are the world leaders. I’m not going to dispute that, but I’m also not going to name them. They don’t exactly need the extra publicity. It’s worth noting that any decent recovery firm would have reached the same results from the batch of damaged drives.
I do have a couple of problems with the way some of the drives were “destroyed.”
Sledgehammer. This would have been a good way to destroy a drive, but only if it had been removed from the PC first. Effectively the metal PC case acted like armour, thus protecting the drive from the brunt of the impact.
Tractor. Same as above. If the drive was bare, and on solid ground, then maybe the tractor would have done more damage. Instead, the PC case protected it sufficiently and all the data was recoverable.
Golf Swing. This was great in the example shown, but is a bit unreliable. If you only hit the edge, or if the disk didn’t have glass platters then it may have been recoverable. Maybe take it apart first, then you can see if it’s damaged.
Tea Damaged USB Pen. This was a good one. Solid state storage should survive liquid damage, as long as it is powered off at the time. When dried out, there is a good chance of getting the data back. The worst thing you could do is plug in a wet drive, as this would cause an electrical short, and potentially damage the electronics of the device, and even the computer you plugged it into.
Big Magnet. This was a good one, and surprisingly effective. Only problems are the fact that most people don’t have a giant magnet, and unless you test it afterwards, you wouldn’t know if it had worked.
Toaster. This is an interesting one for me. Of course the toaster damaged the PCB (circuit board) of this hard drive. These drives were quite old, so that was no major problem. If however these were more modern drives the story could have been quite different. A lot of newer drives encrypt the data using keys stored on the PCB. If you melt that PCB, then you have a very difficult job on your hands.
Torched. 100% successful. If you can see the drive destroyed, then that’s perfect.
Liz later made some good points about the reliability of CD / DVD storage. I agree that although the quoted life spans of DVDs are enormous, in reality DVDs often only last for a couple of years. We have had discs in for recovery that have been stored in temperature-controlled server rooms that have still failed well short of their estimated lifespans.
Hard disks can last for ages. We have some here that are well over 15 years old and still going strong. The problem is that they can fail without any warning. It is sound advice to backup one drive with another, and then another. This is the only surefire way to avoid being stung by a failed drive. Dallas made a good point of moving one of the backups off site, which is also a good idea.
I didn’t like the scrambling advice given near the end. There are problems with the way hard drives are designed, which can prevent the software from accessing bad sectors, and hidden parts of the disk. Although only small parts of the disk, you could leave enough data there to be targeted by fraudsters or whoever.
I advise a two pronged approach. First erase / scramble the data, then physically destroy the drive. This makes it far less likely that your data could end up in the wrong hands.
It is good to see this sort of thing on mainstream TV, and the advice given was a good starting point for most people. Despite my points above, it was basically a good show: Interesting and informative, with a decent amount of good info.
Many people have little or no knowledge of the way their data is stored, so any way to bring this to their attention is good in my books.
We needed a dual boot Ubuntu and Windows 7 setup today, which had a few stumbling blocks.
Problem 1. – GPT Partition
Installed Windows 7 on half a 500GB drive and left 250GB free for Ubuntu. When booting to install Linux, it didn’t see Windows 7 due to the GPT partition that Windows 7 uses by default.
Used Gparted from the Linux live cd to format the drive with an MBR partition layout. (Note:- This destroyed the original Windows installation.) Then installed Windows back to the drive. This time, Ubuntu saw Windows 7 during installation and was happy to install alongside it.
Problem 2. – Default Windows Dual Boot
Although the machine was now happily dual booting, it would default to Ubuntu, regardless of which changes i made to the grub config. Startup manager would make changes to grub, but it would still default to Ubuntu.
Nariub on the Ubuntu forums suggested changing the os-prober number, so it loads that first. This puts Windows at the top, making it the default. Perfect for what we needed.
The command for this, which worked for us on Ubuntu 11.10 & Windows 7:
My brother has just brought me his laptop to look at after forgetting the login password. It was frequently asking him to change the password, and one day he changed it and then forgot it. I found a simple command to stop the password from expiring:
First run Cmd (Command Line) as Administrator (click Start -> and type cmd. Right click on Cmd and choose “Run As Administrator”). If you followed correctly this should give you a black command line window with white text.
net accounts /maxpwage:unlimited
And press return or enter.
It should congratulate you, or say successful (can’t remember the exact wording).
The password should then last forever, or until it is changed manually.
Note: It is good practice to change passwords regularly, however outside of corporate IT land can be a huge hassle. Just ask my brother 🙂
This Seagate ST31000333AS 1TB drive suffered a head crash resulting in severe media damage. The contamination caused by the damage can be seen on the disc surface as well as the internal filter. The filter is usually white but is now silver metallic in colour as a result of the contamination. The customer did not have any indication that this was going to happen.
So you have a Mac which boots up to a desktop as normal, but without your recovered data. This is not quite ideal, and gives us a few things to sort out before the migration. If you’ve not yet had your data recovered, check our Mac Data Recovery Services.
You will first have to backup any newly created data. If something goes wrong with migration then you don’t want to lose your new data. If this new system has been created with the same username as the old one, you will not be able to import the old user without renaming which is not advised. After you have backed up your files, you could open “System Preferences / Users & Groups,” (“Accounts” in pre Lion systems) and rename the current user to something else. This will allow you to transfer your original user account and Applications into the correct locations on the new system. If your system is set to automatically login to a user account (ie. doesn’t require a password when you boot) then you will need to turn off “Automatic Login” setting under Users & Groups to allow you to access the old user account.
Fingers crossed you should be ready to migrate.
Connect the new hard drive to your Mac. If it’s a desktop 3.5” drive then plug in the power adapter and switch it on. If you are restoring to a laptop then it would be a good idea to have the AC adapter plugged in, as this can take a while.
Go to “Applications / Utilities” and launch “Migration Assistant”. Choose “From another Mac, PC, Time Machine Backup or other disk.” Then choose the second option; “From Time Machine Backup or other disk.” You should see the orange icon for the external drive, labelled with your job number. Click on it and then click continue.
You can choose to migrate everything, or be a bit selective. You cannot choose individual files to migrate, only whole user accounts, Applications, Settings, and other files. Make your choice and click next.
The migration itself can take a while depending how much data you have. Once complete you can boot into the Mac and it will feel very familiar. As if nothing ever failed. If you backed up any files from the new system then now would be a good time to load them back on.
Migration Assistant generally does a good job of transferring your data and software. If any software installs files into the Mac system in unusual locations it may need to be reinstalled, but most Applications like Microsoft Office and Adobe Photoshop will be transferred correctly.
Raw data is what we get when we recover files without their folder and file names. Instead of a My Documents folder, with photos and documents arranged into separate folders, what we get is a folder named JPEG for example, with thousands of consecutively numbered jpg files. The same with office documents, you would get a folder with thousands of doc, xls, docx or xlsx files.
These raw files will be fully usable and contain all the same info that they did originally. You will still be able to open them, edit them and save them, they are just unnamed.
Why is this data in RAW format?
When we recover data, we always prefer to get it back in the original structured form. When data has been deleted from a Mac, or when a hard drive has been reformatted and then partially overwritten, it can be impossible to rebuild all of the data in structured form, as the structure has been overwritten or damaged. This is when we opt for RAW files.
With a RAW recovery, what we are basically doing is searching the whole hard drive for files in known formats. This means we usually get a lot of office documents, jpg images, photoshop psd files and some others. If we need to find an unusual file type then we need a few sample files to be able to generate the correct scan info. RAW recovery is not always possible for every type of file. An example is Apple Garageband project files, which are actually just folders with the name .band on the end. On the mac, these folders are treated as packages, with folders and files inside that you don’t usually see, (If you right click one and choose ‘Show Package Contents’ you will see what I mean,) but for the purposes of RAW recovery we cannot get back those files. (We would however get back the RAW AIFF files and recordings from within the projects. It’s not ideal but may be better than nothing.)
What to do with the RAW files
If there is only a small number of files, then you can manually open them all up, see what’s inside and then rename them to something useful. Luckily, for certain file types, there are other ways to make sense of them. It’s called meta data, meta tags or EXIF data.
This meta information is stored inside the files, so even if the file and folder names are lost, we still have the tags.
A brilliant piece of software called Amok EXIF sorter will plough through thousands of jpg files, read their date tags and then place them in dated folders. It can also do other fancy things with tags, but default setting will create a decent structure.
For music, iTunes or any other music manager will usually rename the files in the library based on the artist and album tags. In iTunes, just make sure it is set to: ‘Keep iTunes media folder organised.’
Document files such as doc, docx, xls and xlsx also have some useful tags that we can use to make sense of the masses of numbered files.
In Windows, set the View to ‘Details’ and you should see a series of headings such as Name, Date Modified, Type, and Size. Date modified will show the date that the files were recovered so is useless for this task, however if you right click on this heading you will see other available columns, with ticks beside a few of them. The ‘More…‘ option at the bottom contains loads of tags that we can use to sort the data.
Good headings for office documents are Author and Date last saved. You can experiment and see if any of the other tags are more useful to your specific data.
Good headings for jpg files are Date taken (Date picture taken) and Camera model. Again, there are others which may be useful.