How to Develop a Digital Negative file

Develop a Digital Negative (DNG) file out of your camera raw files

Shooting in RAW is extremely popular as the data is taken directly from the camera sensor, which offers more creative flexibility in post-processing. In comparison to RAW, RGB data is usually the result of a lossy image processing pipeline such as demosaicing, sharpening, noise reduction and compression. Access to RAW data gives you the opportunity to use your own image processing pipeline for maximum image quality.

However, a lot of camera manufacturers have proprietary RAW files, whose specifications may not be available for the public. This leads to a restriction of the creative work flow as sharing the data between post-processing software might not be supported.

Adobe proposed and offers an open, lossless raw image format based on the TIFF/EP standard format called Digital Negative (DNG). The specifications of the newest version can be found here. Using the Adobe DNG converter, the RAW files of most of the camera manufacturers can be converted to DNG. However, this tool can not support all type of camera RAWs due to the variety of manufacturers and specifications. Consequentially, if you use a camera which does not support saving images in DNG and Adobe does not support your camera type, you have to convert your RAW files to DNG by yourself.

In this article we present two ways for the DNG conversion. The first way is to build and use the DNG SDK from sources, provided by Adobe. The second possibility is to make use of the fact that DNG is an extension of TIFF/EP and utilize LibTIFF instead.

DNG SDK

In order to compile the DNG SDK on Windows with Microsoft Visual Studio newer than 2013, a few tweaks are necessary.

At first, the Adobe’s Extensible Metadata Platform (XMP) library has to be built. Copy the whole folder to the root directory of the DNG SDK and rename it to xmp. In the build folder, there is a batch script and a shell script, namely GenerateXMPToolkitSDK win.bat and GenerateXMPToolkitSDK mac.sh. Execute the script that suits your operating system and your architecture. Double check that all third party libraries are included and linked properly in the project configurations.

After XMP was built successfully, download the second library which is needed for the DNG SDK called libjpeg and unpack the downloaded archive in the libjpeg folder. Now head on to dng sdk/projects/yourOS/ and open the project file. We will build the dng sdk project, the validation tool is prebuilt and available here. Make sure you define in the preprocessor the following values qDNGXMPDocOps=0;qDNGXMPFiles=0;qDNGValidate=0;.

For building in debug configuration with MSVC, a small patch of the dng std allocator class is necessary, on line 564 of dng memory.h subsitute qWinUniversal to true. Now everything is set up and ready to build. The output libraries, i.e. dng sdk x64 Debug.lib and dng sdk x64 Release.lib will be created and the SDK is ready to use.

The most important objects to create a DNG from scratch using this SDK are the following:

  • dngHost : Initial point of contact between host application and DNG SDK
  • dng pixel buffer : Specifies the layout and buffers our raw data
  • dgn image : Where our image buffer will be put in
  • dng negative : Providing functions working with a digital negative
  • dng camera profile Setting colorprofile, e. colormatrix mapping XYZ to the color space of our camera
  • dng image writer Writing DNG image to files

LibTIFF

The second possibility to produce a DNG is to make use of the fact that DNG is an extension of TIFF/EP and uses the C-library LibTIFF. This library, which is  also cross-platform, is much easier to build than the Adobe DNG SDK, but does not let you have absolute control over the Digital Negative, i.e. using Metadata different than Exif would require an additional library. Only CMake and MSVC are required to build the sources on windows. Run CMake and build the INSTALL project.

In order to produce a DNG, we have to create a TIFF-pointer with TIFF *pTiff = TIFFOpen(“yourFileName.dng”, “w”). Similar to the DNG SDK, necessary information like image size, bits per sample, sample format, image orientation, bayer pattern, colormatrix has to be set. In order to do this, libTIFF offers functionalities to set the tags directly e.g. TIFFSetField (pTiff, TIFFTAG CFAPATTERN, ” 01 02 00 01″); which sets the bayer pattern to GBRG.

Here is a table of the most important ones:

  • TIFFTAG IMAGEWIDTH, TIFFTAG IMAGEWIDTH : Image size
  • TIFFTAG ROWSPERSTRIP : Organize image data to strips for faster I/O access
  • TIFFTAG BITSPERSAMPLE : How many bits represent one pixel
  • TIFFTAG SAMPLEFORMAT : Usually given in unsigned integer, important for storing in float is desired
  • TIFFTAG COMPRESSION : Compression scheme
  • TIFFTAG PHOTOMETRIC : Photometric interpretation, in this case we read data form color filter array
  • TIFFTAG CFALAYOUT : sptial layout of the color filter array
  • TIFFTAG CFAPLANECOLOR : Colors of the color filter array, red green blue for example
  • TIFFTAG CFAREPEATPATTERNDIM : Size of the bayer pattern
  • TIFFTAG CFAPATTERN : Specifies the bayer pattern

 

  • TIFFTAG SAMPLESPERPIXEL : Samples per pixel
  • TIFFTAG ORIENTATION : Orientation of image, usually coordinates starts at the top left corner
  • TIFFTAG PLANARCONFIG : Specifies if we have a single image plane or seperate planes of data
  • TIFFTAG DNGVERSION : DNG version number, default: 4.0.0
  • TIFFTAG DNGBACKWARDVERSION : DNG compability version
  • TIFFTAG COLORMATRIX1 : Transformation matrix from XYZ to camera color space
  • TIFFTAG ASSHOTNEUTRAL : Selected white balance in camera color space

For feeding in the image buffer row by row we can use TIFFWriteScanline (pTiff, pointerToFrameBuffer, row, 0);. Clean up your resources and there you have it! Your self-created digital negative.

dngsaved

Figure 1: Digital Negative Sample Image                                                 

snipJpg

(a) Snippet of JPEG                                                                                  

snipDng

(b) Snippet of DNG                                                                                 

Figure 2: Detail comparison of the same shot between JPEG and DNG. The digital negative is richer on details and has less debayering artefacts than JPEG