The Internet is all a-flutter about this announcement Greg Yaitanes made via Twitter that the House, M.D. season finale (airing next month on FOX) was shot entirely with a Canon EOS-5D Mark II. This makes the 5D Mark II the first video-capable digital SLR camera to ever have been used to film a “prime time” television episode.

Yaitanes, who is a co-executive producer and director of House, M.D., said that they chose the 5D Mark II because they wanted to be able to fit the camera into “tight spaces,” they used mostly Canon prime lenses as well as the 24-70 and 70-200 zooms, and that no brackets or stabilization tools were used. I can’t wait to see how the episode looks!

The video was recorded onto large CF cards (Yaitanes said “18gb,” but I think he meant 16 GB), which afforded only 22 minutes of shooting before tossing in a new card.

You can read PetaPixel’s summary of the Q&A that has taken place so far via Twitter for more luscious details, and read engadget’s coverage as well as Gizmodo’s coverage. If you ladies and gents who are technology-inclined don’t already check out those two blogs on a regular basis, you may want to add them to your news reader of choice.

That is the question.

Whether ‘tis nobler in the mind to wrangle the proprietary formats of your camera manufacturer, or to take arms against a sea of sidecar files, and by opposing, end them…

But enough pseudo-Shakespeare for one post.

There has been some chatter on the Interwebs lately concerning the DNG format: there are quality and archival concerns, whether it’s worth converting one’s entire library to the format, what the benefits might be, and whether one ought to care. Today, I weigh in. For what it’s worth.

Coincidentally, I’ve used the same post title as Matt Kloskowski did in his take on the subject on Lightroom Killer Tips. Matt didn’t add a pseudo-Hamlet line, though, so I feel like I’ve done the headline proud.

After the break, a complete rundown on DNG; trust me, it’s going to be technical and editorial.

What Is DNG?

Skip this section if you already know what DNG is, or read it if you want to know what I think DNG is.

DNG is the “Digital Negative” format, spearheaded by Adobe. It is an “open” format, and a “standard” in some sense, though not an official ISO standard (yet). When we say that the format is “open,” we mean that the precise contents of a DNG file, its byte offsets, containers, methods of generating and reading the files, and so forth, are publicly available. This is not the case with any other RAW file format out there. The ability to read and write manufacturers’ RAW files (e.g. Canon, Nikon, et al.) is either licensed from the manufacturer or reverse-engineered.

Adobe created the standard out of, apparently, a desire to “universalize” (if you will) the ubiquitous RAW file formats that all manufacturers have created for themselves, and to introduce a truly interoperable format that any software or hardware maker could employ without 1) wrangling many different formats at once, or 2) singling out particular users of particular products¹.

The DNG format is based on many existing standards owned by prestigious “standards development organizations” (or SDOs), some of which you’ve probably heard of; they go by the names TIFF, JPEG, XMP, IPTC, ICC, CIE, and ZIP, to name a few. So, although the DNG format itself hasn’t been embraced by an SDO such as the International Standards Organization (ISO), it comprises previously standardized formats and is only a snippet of red tape away from being SDO-backed.

You can read a lot of nitty gritty on the format itself explained by Barry Pearson on his DNG Specification page.

Why Would You Want to Use DNG?

There are three main reasons that are generally cited:

• Archivability (future-proof-ness, if you want)
• Interoperability (openness, we like to say)
• Efficiency (YMMV, more on this later)

Archivability (which I’m pretty sure isn’t even a word) means that fifty years from now, when Canon owns the world and Nikon is completely extinct (har har), flying cars are a reality, and everyone has an army of robotic servants, your DNG files will still be supportable by manufacturers of hardware and software alike, because no secrets about it have been kept from the public.

“[I]f Nikon decides to not support my raw files one day, there’s some 15 year old in his room that’ll code up a raw conversion program in his sleep.”
—Matt Kloskowski

Does this claim hold water? I don’t know, you can download free software right now that can read the Photoshop version 3.0 format, which is already decades old, so it seems to me that if the ability to parse a format is out there (even if it was obtained semi-legally or with much effort by third-parties), it will be difficult to lose it. Moreover, a format such as Canon DNG or Nikon NEF will likely remain supported by anyone you care about, or, as Matt Kloskowski put it, “…there’s some 15 year old in his room that’ll code up a raw conversion program in his sleep.” So I am not worried about losing access to my RAW image data.

Interoperability means the ability for you to take the same file and use it in many different places. Of course whether this is an advantage to you or not depends on the places where you want to be able to use your RAW files. For me, Lightroom is the only program on my entire computer that ever sees a RAW file. I suppose if I round-trip through Photoshop, Lightroom is going to pass the CR2 through Camera Raw, but we’re talking about two closely-integrated Adobe applications there; whatever formats one supports, the other is sure to. Photomatix is going to see TIFFs, on the web you’ll see JPEGs, etc.

So why does interoperability matter? Adobe’s point in creating DNG is that it may matter in the future. If Great New Software X decides they can’t support your camera’s RAW format, and if you’ve converted it to DNG already, well, problem solved. Because Great New Software X will certainly support DNG given that the format is completely open and drop-dead simple (not to mention free) to implement, you have a much better chance of being able to drop your existing DNG images into any new, shiny tool.

“I actually tried ZIP compression in prototype versions of DNG, but the compression ratio was much better using lossless JPEG.”—Thomas Knoll

Okay, what about efficiency? This is where things get kind of cool. The Adobe DNG format stores the actual pixel image data in what is called Huffmann lossless JPEG format. What that means is that DNG files can sometimes be as much as 20% more efficient at storing image data on disk than a comparable RAW format, thus DNG files may be as much as 20% smaller. Lossless JPEG is completely pristine; there is no image-altering compression done, so the data is totally preserved, albeit compacted. The Huffmann algorithm for this compression happens to be more efficient than ZIP when there are more than 8 bits of data per channel (RAW is 12 or 16), so the Huffmann algorithm was used.

But here comes the kicker… Canon’s RAW (CR2) format already uses Huffmann lossless JPEG for its internal image data storage. So if you shoot Canon RAW, you will see no increase in data compression, since the data is stored in exactly the same way already. If you shoot in Nikon RAW (NEF), you will immediately see a 20% decrease in file size and no change in image quality.

For non-Canon shooters, that’s probably the most compelling reason to convert to DNG right now, which I should point out Lightroom can do automatically for you at import time. You’ve seen the option, right? There are a couple of ways to convert to DNG in Lightroom, and there are three of them explained on TheLightroomLab.com.

Sidecars Aren’t Just for Kids

No, seriously, most motorcycle sidecars could definitely seat an adult…

Joking aside, what we’re talking about are metadata “sidecar” files, typically named something like IMG_0195.xmp and so-called because they are saved alongside your original RAW image files, like the sidecar on a motorcycle. Sidecar files have been around since the invention of metadata and metadata libraries. The XMP format for storing image metadata in a sidecar file was developed by (guess who…) Adobe. XMP is necessary because the metadata support within RAW file formats and other imaging formats may comprise only a subset of what, for example, Lightroom is able to save and search.

Certain formats such as PSD, TIFF, and JPEG have pretty flexible metadata support in them already. DNG is no different. The “sidecar bonus” of the DNG format is that when you’re using DNG, you don’t need sidecars.

But who does, anyway? The fact is, Lightroom stores all of your image metadata in its own catalog. This is done 1) to make it quickly searchable and editable, and 2) to centralize it. If you want to write metadata to disk for certain files (or the whole catalog), then Lightroom will decide, based on the source format of each image, whether to save it directly into the file or to create an XMP sidecar file.

The purpose of XMP is to give your image metadata a place to live when the image file itself can’t accommodate it. So why would you need this ability? Two reasons. (Assuming your RAW files are in NEF, CR2, or similar; non-DNG).

1. If you share images with other people who use Lightroom and you want them to be able to see what you have done in the Develop module, as well as the IPTC tags, you will need to send them XMP sidecar files. This is preferable to sending your entire Lightroom catalog or creating a new one to house only the images you are sending.
2. In the event of a catastrophic disaster where your Lightroom catalog backups are corrupted, you can restore 95% of your data using the source RAW image files and associated XMP sidecar files. What you would lose is virtual copies, your history, and any collections. All edits and other settings are stored in the XMP files.

There are some major caveats here, though.

1. When you’re sharing images, you can always export them to DNG, which makes XMP unnecessary.
2. In the event of a catastrophic disaster, you’ll only have XMP files if you’ve previously selected batches of images within Lightroom and triggered the “Export Metadata to Files” function. Presumably you would carry out this procedure on images you have finished editing as part of your backup strategy. For the record, I do not do this.

Conclusions

So where does this leave us as far as DNG goes? Should we all run back to our computers right now and convert our whole libraries to DNG format? Once again, here are the major advertised benefits of the DNG format:

1. Archival (future-proof)
2. Interoperable (widely compatible)
3. Efficient (storage-wise, at least)
4. All-inclusive (obsoletes XMP sidecars)

It seems to me that the “archival” and “interoperable” characteristics of the format are bonuses, and as a supporter of free and open standards, I tip my hat to Adobe simply on principle. These characteristics don’t, however, increase the immediate convenience or reliability of my workflow, nor do I think the archival nature of the DNG format will have a measurable impact on my workflow in the coming years.

Since I am a Canon shooter, my Canon RAW (CR2) files already compress image data using the same Huffmann lossless JPEG system that DNG does, so I get no benefit there. If you use Nikon, et al., you may see an immediate storage benefit from converting your library to DNG.

Perhaps the most compelling reason to convert your library to DNG, or to start using DNG for your imports going forward, is the fact that DNG files are capable of storing all Lightroom-specific metadata directly within themselves, without the use of XMP sidecar files. If you are diligent in writing metadata to the files when you are through editing them, those files then fully encapsulate all the work that you’ve done, in one place, suitable for backup.

Even in the event that you lost your Lightroom catalog, you could still restore the final, edited versions of the images (which is where all of your hard work goes, after all), as well as tags and other helpful metadata. The only things you’d lose are virtual copies, history (of limited long-term utility, anyway), and collections (which can be rebuilt without nearly as much work as re-developing all your images).

To DNG or not to DNG? I think I will.

1. Like yours, for example.

This time, though, a piece of “news” hit my screen that I had to share, and I hope that by reading it here you will receive the whole story and not just 1/3 of it as some people did when they got the information from digg, reddit, or similar.

We’re talking about megapixels, and we’re talking about marketing, and we’re talking about lies. Lies perpetrated by the manufacturers of point-and-shoot cameras with tiny little CCD sensors who keep increasing the megapixel count and marketing it like it’s the end-all be-all of imaging performance.

Not so! Read on…

This issue surfaced when a nameless blogger (literally, I have no idea who this person is) posted an article (you might call it a “diatribe”) about the “suicidal march” of point-and-shoot cameras toward more and more megapixels (spurned by the Consumer Electronics Conference, or CES, which was earlier this month in Las Vegas), and the results of cramming lots of pixels into a small area. You can read that article here.

In order to understand some of the background, though, you’ll want to read his or her other article about optical diffraction and Airy disks, which is very interesting, and then read his or her follow-up article where he (or she) addresses some of the anonymous Internet’s criticisms.

Now, I know it’s a gamble to spread anonymously written blog posts around as though they’re primary sources and laud the nameless, faceless author for their courage in uncovering what seems to be a pervasive scam, but to be honest I was blown away by the apparent truth of the whole thing.

Though my question has little to do with your most recent article, I find that the expertise you’ve shared since I began following your site to be compelling and hope that you can provide me an answer that will serve my needs. I have seen many digital photos over the years, some pretty dismal and some pretty spectacular. My question was born out of seeing, for the first time, an exhibition comprised of 150 works by Ansel Adams, which were nearly all “Silver Gelatin” prints. My question is (setting aside for the moment the composition, line, form and majestic beauty of many of the locations) can any digital print be made in such a way that a knowledgeable observer would not be able to distinguish it from a print made using the silver gelatin method?

In the interest of full disclosure, I should probably mention that the comment was posted by my father, who, having a degree in fine art and a gallery of his own, takes a vested interest in some of the topics I discuss. That said, I thought it was a very relevant question and one that many of you might have thought about, too, so here’s what I think.

First, it would depend on how knowledgeable the viewer was and how close they could get to their subject. Second, there are many technologies available, so the short answer is “probably,” but the long answer is more interesting.

So-called “lightjet,” which combines a digital (laser-based) exposure of photographic paper with traditional (chemical) development methods, produces very fine results, especially of full-color images. Upon very close inspection, however, it would be clear that there are many colorful dots making up the print. Still, these prints have fantastic longevity, are often quite lustrous, and are available for a lot less money, comparatively, than other methods. An added benefit is the ability to print on any brand and type of photographic paper available to photographers, from Kodak Endura to Fuji Crystal Archive. There is a certain je ne sais quoi surrounding real photographic papers that may be the strongest argument for lightjet.

Inkjet prints boast a longevity nearly comparable to traditional development and are capable of a much broader range of color than lightjet. The highest-end inkjet printers now deliver between seven and twelve physical inks in picoliter droplets that mix on the paper to create a continuous tone image. The ink droplets are dispensed by a piezoelectric system and can be either dye-based or pigment-based, each having their own archival and color properties. Inkjet printers, however, are much more expensive to run than ordering your prints from a third-party photofinisher (who probably uses lightjet), and if you need the highest quality available, you will have to buy, configure, and operate the setup yourself, which is no small task!

Giclee (zhee-clay or gee-clay) has also held its own against the influx of lower-cost inkjet solutions, boasting the ability to print on materials such as canvas and at resolutions beyond what inkjet or lightjet typically can achieve. Giclee (sometimes called Iris printing because one of the original models was called Iris) is essentially a CMYK inkjet system, meaning that only cyan, magenta, yellow, and black inks are used, though I have heard of giclee printers that use six inks. The ink is fired from glass nozzles at one million droplets per second and each drop is electrically charged so it can be directed toward or away from the paper by electromagnetism. The paper itself is affixed to a drum that spins at about 180 inches per second. Giclee is probably one of the most mechanically impressive printing methods around.

Artists have chosen giclee for years because of its faithful color reproduction and ability to print on “artistic” substrates such as canvas. A single giclee print, however, can cost $50, $100, or $200 to produce, not including the calibration and other services required to achieve the results you need. Giclee is more favored by painters than photographers.

Four-color offset lithography, which is how all print publications are produced nowadays, is actually capable of near-giclee quality, however American print shops tend to be too traditionalist to adopt the color management methods necessary to produce fine art prints to an exacting standard. Bill Atkinson, a man absolutely fanatical about color accuracy, collaborated with a Japanese print shop to implement color management methodologies for their four-color presses. In return, they printed his book of rock photographs. That book may be the only example of accurate color reproduction through offset lithography on any American bookshelf. The difference between the capabilities of high-end offset lithography and giclee is entirely due to the willingness of the technical staff involved to use modern digital color management methods.

At the end of the day, can any of these digital solutions deliver a result as austere and striking as a pure black and white gelatin silver print? Probably not, but they can come very close. I am convinced that Ansel Adams himself would be a dedicated and outspoken advocate of digital photography and all of its methods were he still alive today.

$random:left$

My opinion is that digital photography is, and should be treated as, entirely comparable and identical to traditional photography. Although the differences in process and technique bear investigation for the ambitious viewer and may play a role in serious competition and critique as parameters for comparison, they have no more bearing on the effect of the work as would the brand of pencils used by an illustrator.

In the field of art, competition and formal review should command the examination of an artist’s basic choice of media and execution. While even the casual viewer can deepen the experience of a piece of art with knowledge of its process, that information is only helpful insofar as it expands appreciation or increases understanding; all artwork is comparable given only the experience of it intended by its creator.

To give an example of what I mean, consider two paintings. Each painting was created with different types of paint and different brushes, one on canvas and the other on paper, one representational and the other abstract, one lacquered and the other not. Can these two works of art be compared objectively as paintings? My gut says they can. The majority of casual viewers would never seek out these differences because the two works can be compared and contrasted, fairly and completely, simply on the basis of their appearance.

$random:right$

Similarly in photography, two photographs may appear very much the same but may have been created in vastly different ways. In traditional photography alone, a myriad of chemicals and materials are available to the artist. Comparing two photographs, one traditionally printed and the other digitally created, should pose no problem to the viewer; only in formal competition and critique should this piece of information carry any weight, and that is also true for the painting example given above.

Digital photographers find themselves, in many cases, having to defend their work on the basis of its production methods rather than its artistic integrity, and it saddens me. When, in the history of art, have its methods come under such scrutiny? Producing new work in new ways has always been a core value of the field of art as a whole.

$random:left$

What is saddening is not that a photographer would spend time and energy defending his or her craft, for that seems to me a noble endeavor. What saddens me is that anyone would spend time and energy criticizing digital photography, in particular, on the basis of its process rather than its results. Few other fields of art come under such scrutiny; perhaps because many fields of art are less commercialized or because they occupy areas of the art world not very well traveled by the mainstream. Whatever the reasons may be, I hope that the future brings a greater acceptance of digital photography as a photographic methodology, not to be treated differently than the many ways in which light has been captured and reproduced in the past.

Pure photography is defined as possessing no qualities of technique, composition or idea, derivative of any other art form. The production of the “Pictorialist,” on the other hand, indicates a devotion to principles of art which are directly related to painting and the graphic arts.

Group f/64 was conceived in explicit opposition to the Pictorialist movement, which “subscribed to the idea that art photography needed to emulate the painting and etching of the time.”² Quite to the contrary, Group f/64 believed very strongly that photography “must always remain independent of ideological conventions of art and aesthetics that are reminiscent of a period and culture antedating the growth of the medium itself.”³ These are two very different viewpoints. One sees photography as a medium through which to create works representative of the predominant aesthetics and style of other types of art at the time, and the other sees photography as a new medium with its own aesthetics and style that should be preserved.

Purists almost by definition, the members of Group f/64 sought to stretch the boundaries of photography through strict adherence to its core methods. They used as few image-altering devices or techniques as possible; no lens filters, no exotic darkroom processes or equipment. Their images aspired to a crisp, infinitely-focused, tonally brilliant standard upon which many future photographers would base their own explorations.

It should be noted that although Ansel Adams used extensive, complicated darkroom techniques on many of his most famous prints, he was both an advocate of smart, precise post-processing as well as maintaining the integrity of the medium by minimizing distortion of the subject, and it was likely for the latter reason that he helped to form Group f/64.

But that was 1932. The “ideological conventions of art and aesthetics” of 1932 have been entirely replaced in the age of the computer. Almost in parallel to the 1932 Pictorialist/pure photography dichotomy, there are those who see digital photography as a mere convenience; a new, faster, and in some ways more inexpensive way to maintain similar aesthetics to photography of the past, and there are those who see digital photography as an entirely new medium.

I believe that both viewpoints are correct. However, digital photography certainly brings with it a veritable cavalcade of new capabilities and equipment, inheriting credibility and respect from its traditional, silver halide forebears, but independent from them in all other ways, both technical and aesthetic.

That being the case, what now embodies the “qualities of technique, composition or idea” specific to “digital photography?” If a new group came about with the same goals as Group f/64, but updated for digital photography, what would its major tenets be? With such extensive editing capabilities in the hands of even the beginner through Photoshop, Paint Shop Pro, The Gimp, iPhoto, et. al., it goes without saying that manipulation of the image is going to play a role. Whether it is basic tonal adjustment and sharpening, or more drastic and potentially destructive edits, such changes fall comfortably under the umbrella of digital photography as a medium.

When Ansel Adams stood before the St. Hernandez valley on that day in 1941, he could see in his mind the process that would be used to bring the exposure to life, from development to printing. It is that mastery of craft that has perhaps been forgotten now that the capacity to make hundreds, if not thousands, of exposures is available. This is one impact of the digital revolution. Scarcity encourages innovation; when there is less to work with, more attention is paid to planning and execution to squeeze every drop of that creative juice out of the moment.

Likewise, abundance breeds laziness. It’s too easy to snap 100 exposures of a subject in the hopes that one is in focus when each exposure costs you nothing and with cameras capable of several exposures per second.

This article doesn’t mean to draw conclusions. The ratio between excellent photographers and poor photographers is likely to be much the same today as it was in Ansel’s time (counting only those who consider themselves serious hobbyists and amateur professionals; photography is no longer exclusive to the exceptionally passionate and the exceptionally wealthy.) Still, digital photography raises a lot of interesting questions and only time will tell how it will be treated by the art historians of the future.

1. Wikipedia, Group f/64
2. Wikipedia, Pictorialism
3. ibid.