|
Search this group's discussions
|
Delete the RAW (NEF) file after conversion to DNG?
|
Do you delete the original RAW (in my case NEF) file after you have converted to DNG. In order to save some HDD space?
Posted at 7:39PM, 21 December 2005 PDT
(permalink)
|
|
I do. I can't see any reson to keep the NEF RAW file. Of course, I've been known to be wrong about these things before...
Posted 78 months ago.
(permalink)
|
|
I don't
Originally posted 78 months ago.
(permalink)
tychay edited this topic 78 months ago.
|
|
I'd never delete the RAW file, it's the original. Would you throw away your film negative?
Posted 78 months ago.
(permalink)
|
|
|
Anders, this is my question, DNG claims to be a longer lasting hardware independent negative. And if DNG is going to be my long term format, then why should I keep the NEF. This was my thought, was wondering if that view was shared.
Posted 78 months ago.
(permalink)
|
|
|
tchay... now I'm more confused.
Posted 78 months ago.
(permalink)
|
|
|
@Gordon: DNG is future-proofed and hardware-independent. It is also a very simple specification to implement (it's just TIFF spec with a standardized metadata format). It is because these advantages that it is useful. Imagine if your camera comes from a vendor that might go out of business, or Canon decides to stop supporting your old digital camera, or your image editing tool is made by someone really lazy.
But hardware independence is a BAD THING(tm). An NEF converted into a DNG has the unique camera data as a string (like "Nikon D50") in the header field. For all intents and purposes, that file is identical to the Nikon D70s NEF converted to a DNG, other than that string. Applications that process and handle DNGs will handle images from both cameras identically. It will handle images from different lenses identically. It will handle almost any image from any digital camera nearly identically. DNG is an attempt to treat all images almost identically.
(There are some arbitrary parameters in the DNG spec such as “anti-alias filter strength” that can make a difference in processing, but these are qualitative numbers, not quantitative and thus aren't camera-specific, aren’t accurate, and aren’t precise.)
The problem is cameras aren’t identical.
The Nikon D50 and Nikon D70s RAWs are not the same and shouldn’t be treated as such. The anti-aliasing bayer microlenses placed in front of the sensor and the exposure system are different between those two cameras. Nikon Capture 4 knows this and treats them with a slightly different recipe when processing. DxO Optics Pro does the same and even adjusting for lens defects and lens-body combinations.
You need the NEF file to take advantage of either program. Can you reconstruct the NEF file from a DNG file? No.
Sure, replacing your RAW files with DNGs is a view that is shared. It is shared among people who think that a RAW file format is no different than a TIFF with metadata who “rave” about how “great” Adobe Camera Raw is.
Among those of us who know better, doing so is just buying into some Adobe marketing bullshit.
DNGs are future-proofed and hardware-independent and that's a great thing. They are not your digital negatives and should not be treated as such.
Originally posted 78 months ago.
(permalink)
tychay edited this topic 78 months ago.
|
|
|
Interesting discussion. Thanks for the detailed explanations, tychay.
So, would the general recommendation be to kepe the NEF as a 'true' digital' negative, but also save a DNG copy as a kind of 'just in case' back up? Or would you argue that converting to DNG is essentially pointless?
Posted 78 months ago.
(permalink)
|
|
I convert to DNG directly from the memory card, then after I've taken a copy of the DNGs on a 2nd disc and checked that the conversion and copy has worked, I am able to reformat the memory card and discard the original raw files. I've been working that way without problems for more than 6 months. (For more than 8 months before that, I used DNG, but kept the original raw files as well). I don't suggest other people do that - they must make their own informed decision. The downside is that some prooducts don't (yet) accept DNG files.
Originally I used DNG to save disc space. Now I do it because I use ACR 3.x with Photoshop CS2, and this can save a lot of extra metadata within the DNG files. Name, copyright information, website, phrases describing the shoot, more phrases describing the images, ACR edits and settings, etc. Having all of those within a single file makes things easier. (The metadata can get transfered to downstream PSDs and JPEGs, etc).
The image data within the DNG is just the same as the image data within the original raw file. Instead of talking about "digital negatives", a useful analogy is with 35mm slides. It is the 24mm x 36mm image that is important. You can move it from one slide mount to another without problems. You can write on the slide mount ("add metadata"!) And the raw file, whether NEF of DNG or whatever is like the slide mount, holding in turn the same image.
I'm told that a DNG file holds a superset of the NEF data. It holds the NEF image data, plus the Makernote (moved to DNGPrivateData), plus camera calibration tables which distinguish between camera models with different colour responses, plus other data. But, because it uses better lossless compression, it is typically smaller.
Posted 78 months ago.
(permalink)
|
|
|
Gordon, here's my thinking...
If the RAW data is converted in any way you still need the RAW file if you want the ability to do a different RAW conversion later on.
If the RAW data is stored untouched wrapped in a DNG file you may not have access to all the data later on (some of Nikon's white balance data for instance which is encrypted) unless you can recreate the RAW/NEF file so Nikon's software can read it. Bassically all you get is storage overhead from the added DNG data and you may lose camera-specific functionality.
I am all for DNG but will only use it when my camera/software produces DNG files and when all that data is open to be read by third-party software. Until that happens I really don't see any gain from using DNG, but rather a risk of losing functionality.
Originally posted 78 months ago.
(permalink)
Anders Östberg edited this topic 78 months ago.
|
|
|
@Gilfer: I would use DNGs in place of TIFFs. DNGs carry along the metadata along, TIFF does not.
I would use DNGs if I plan on using Adobe Camera Raw forever in place of any other raw converter. DNGs have the un-demosaiced data so if I think I prefer a particular vendor's RAW conversion, will never change RAW converters, and I make the assumption that any improvements to the converting process will be back-ported to DNG, the the DNG effectively replaces the RAW.
I would use DNG when if my RAW converter has that as an export option. At that point, the RAW conversion has already been done and thus every application that works with DNG will treat it the same. Realize that your DNG file at this point will be 2x your RAW file size which is already 2x vs a highest quality JPEG. Unless you plan on messing with the levels a lot, there is a big hard drive premium for doing such.
I would use DNGs if I think my preferred RAW converter is going to be dropping support for my file. Since software is developed incrementally, this can only occur if I use my camera manufacturer’s converter and they go out of business, drop support for my camera (for no good reason), or drop support for my platform (Macintosh). I can't see this happening in the case of Nikon (which has more revenue than Adobe), but it might happen.
@Barry: You are correct in that DNG data is usually theoretically mappable back to NEF data, but unless you store the NEF bundled with the DNG, you cannot actually get the NEF from the DNG. No such tool exists and the market would not create one.
For example, the Nikon D70 does not store a linear 12-bit/channel mapping of the sensor data. When converted to a DNG, the D70's file contains something like that data along with a mapping function stored in the DNG metadata (or the data already mapped along with the inverse function. I forgot which). In either case, an inverse converter would have to know this detail as well as the NEF file format to correctly reconstruct the D70 NEF in a manner than Nikon Capture could process. Such a tool would have to know all these mapping rules for every version of RAW file out there.
Also, I'd have to look at the spec again, but even though you can store all the metadata along with the DNG doesn’t mean it is always stored. As Anders points out, some data was not mapped correctly from the D2X/D50 NEF. Other data cannot be translated at all yet (D200 NEF). Once I’ve converted and deleted my NEFs how do I recover the data even if I trust that the unknown data is thrown into the "catch-all metadata" part of the DNG spec? That would require a DNG re-converter to know every version of every DNG converter out there.
Nobody is going to write a tool for you to round trip from DNG->NEF->DNG. Because this is the fundamental postulate put forth by Adobe: you should never need your NEF once it has been converted to DNG.
That is based on the mistaken assumption that every hardware manufacturer out there is going to be building their hardware to support DNG. The reality is not one manufacturer supports DNG or OpenRAW currently. Why do you think that is? If there were no strings attached to DNG, a company like Vivitar should at least see a PR boost if they switched over.
Could it be because if they did, DNG would favor a single "one RAW converter to rule them all" (viz. Adobe Camera Raw) despite its mediocrity? There is a lot of qualitative fields in the DNG metadata: one example I gave was “anti-aliasing strength” which is a number from like 0-3. How that image is processed in ACR is a “black box” and if Nikon (say) were to replace NEF with DNG then they would be caught replacing optical and digital AA processing to better match one an arbitrary, undocumented, and “moving target” one by the owner of the “one true RAW converter.”
@Anders: I believe if you check the box that allows you to save the NEF along with the DNG, you can recover the entire NEF using Adobe DNG Converter. (I have not tried this myself.) Of course, this isn’t related to the discussion at hand because the original poster wanted to delete his NEFs to save HDD space and he won’t save any space this way. ;-)
Originally posted 78 months ago.
(permalink)
tychay edited this topic 78 months ago.
|
|
|
Response to tychay:
1. I agree that there is currently no tool to convert DNGs back to NEFs. I doubt if there will ever be a commercial one. But there are some ingenious people out there! There are people who have converted other raw files into NEFs, (Google for raw2nef), and plenty of people who can decode DNGs, so who knows what will happen?
Since there is currently no such product, I would never advise anyone to discard their original raw files unless they were were confident that they would never need to use a product that didn't support DNG. I believe that, by the time DNG is two years old, (end September 2006), there will be few raw-handling software products that don't support DNG other than the camera manufacturers' products.
2. More information about the amount of data stored in DNGs by the DNG Converter has been provided by Thomas Knoll and others. Raw files based on TIFF-EP, with proper use of Makernote, apparently hold all the data from the original raw. The situation improves with each release of the DNG Converter. Here is the situation with version 3.1, (May 2005). (Version 3.2, September 2005, added more, for example the D2X "as shot" WB, and 3.3 beta shows some of the extra preserved metadata for some cameras).
photoshopnews.com/2005/05/23/dng-workflow-part-i/
"If you are shooting with a camera that outputs either NEF’s or CR2’s (Nikon and Canon), essentially, you’ll lose nothing. Starting with Adobe DNG Converter version 3.1 (the one released with Camera Raw 3.1 in May, 2005), the DNG Converter will take all the proprietary metadata from the original file format and safely move it to the private portions of the fully documented DNG file.... Thomas Knoll has clarified what is and is not migrated from the proprietary raw file to DNG files upon conversion. DNG Converter 3.1 and above does indeed move all EXIF private maker note from all TIFF-EP based raw file formats, not just NEF and CR2 files. The file formats that do NOT use EXIF based private maker notes are not migrated. File formats from; Kodak, Foveon and Leaf do not use EXIF private maker notes, so those files are not supported for migration. Additionally, Canon CRW files and Fuji RAF do not completely adhear to the TIFF-EP and EXIF spec. So some metadata will be migrated and some will be stripped. Also, black masked pixels (the far outter edges usually cropped off by conversion software) for certain files, such as some CR2 files are also not preserved in DNG conversion. Future updates to the Adobe DNG Converter will continue to add to the scope of what is preserved and migrated to DNG".
3. 3 cameras and 4 digital backs current use DNG as their native raw format. (Contrary to your statement "The reality is not one manufacturer supports DNG"). See:
www.barry.pearson.name/articles/dng/products_y1.htm
If NO cameras ever supported DNG, and ALL software products supported DNG, I expect many, perhaps most, photographers would adopt a DNG-based workflow. Whether cameras output DNG is the least important factor of all. What is more important is whether the software used by photographers, including that provided by camera manufacturers supports DNG. So far, hardly any of the latter does, and for those photographers, (perhaps a minority), who need to use that software, this matters. (Flexcolor from Hasselblad-Imacon does).
4. DNG certainly does NOT "favor a single "one RAW converter to rule them all""! On the contrary, it will open up raw shooting. That is because it makes the cameras' raw image data much easier to get at by software developers who don't want to suffer the stupidity of having to reverse-engineer the proliferation of undocumented formats we see that the moment. I think what we will start to see are products that treat DNG as their "first class citizen", and if they have to handle another raw fornat, they will call Adobe's DNG Converter via its command-line interface to obtain a DNG file they can work with.
5. Although there are a few fields such as AA whose values are not precisely defined, that doesn't mean they are a "black box". For example, it is easy enough to find out what value Adobe use for the Leica DMR back, the D70, and and so on. (Just look inside the DNGs for those cameras). That provides a set of references. And anyone wanting to output DNG from their camera can simply try various values and see what the Adobe products, (and others), do with those values!
Posted 78 months ago.
(permalink)
|
Would you like to comment?
Sign up for a free account, or sign in (if you're already a member).
|
|
PhotoShop /
Discuss
Feed – Subscribe to PhotoShop discussion threads
