EXIF / IPTC Description / User Comment

[dhackney]

I was faced with a variation of the EXIF description challenge described in a few other posts.

Problem/Bug: BBP not populating EXIF comment/description from IPTC Description
Scope: Problem/bug affects all BBP users who need to replicate IPTC description to EXIF description for JPG images. (This issue may affect other image types as well, I only tested JPG.) Impact is across all images used with any tool or environment, i.e. web photo hosting service, that defaults to or requires the use of EXIF data fields and/or does not support XMP data.

Specific Scenario:
Goal: optimize workflow around DP and BBP
Workflow segment: image preparation for upload to Gallery2
Challenge: Gallery2 limited to EXIF description for image caption

Workflow environment:

Windows: XP Pro, SP2


Workflow tools:

DP: v2.0.4
BBP: v1.7.3.1
Canon Digital Photo Professional 3.1.0.0 (used for RAW conversion)
RoboGEO 5.3.9 (used to insert GPS data when DP cannot find an matching track point)


Test tool environment:

ExifToolGUI: v3.11
ExifTool by Phil Harvey: v7.25
Opanda IEXIF 2.3
Exifer 2.1.5
Adobe Photoshop CS2 9.0.2
Adobe Bridge CS2 1.0.4.6
Paint Shop Pro 9.01



I spent the morning running a series of tests on BBP and came up with the following results.

http://www.hackneys.com/photos/bbp-test/bbp-desc-logic-box.jpg

Full test results and conclusions are here: http://www.hackneys.com/photos/bbp-test/bbp-exif-iptc-test-docs.pdf

Test data set and test suite detailed results are here: http://www.hackneys.com/photos/bbp-test/bbp-exif-iptc-test-data-xl.pdf

Test images are here: http://www.hackneys.com/photos/bbp-test/


Bug Fix / Enhancement Submissions


Clearly label the “Edit Comments” dialog box to reflect its state and purpose, i.e. EXIF Comments.
Clearly label the User Comment / Description displayed in filmstrip and full views as to their source, i.e. EXIF, IPTC (legacy), or IPTC/XMP.
Use the capability demonstrated in conclusion #1 [in PDF document bbp-exif-iptc-test-docs.pdf], i.e. the ability to create and populate an EXIF User Comment / Description field in an image file that contains no pre-existing EXIF User Comment / Description field, to enable population of the EXIF User Comment / Description field from the IPTC Description field on both a manual and @IPTC_description@ batch basis.
Enable manual and batch replication of all relevant IPTC fields to EXIF fields, i.e. owner/creator, description, copyright, etc.

[DavidB]

This is, by any standards, a thorough and impressive piece of work, for which we should all be grateful. I have neither the patience nor all the tools needed to replicate the findings, but I find them credible, and will assume that they are substantially correct.

The work is important, in my view, because BB Pro is one of the few tools that helps users chart a path through through the standards minefield, particularly where metadata is concerned. With its level of standards compliance, BB Pro, when used as 'workflow central', actually enhances compatibility between applications. There is at least one application I regularly use which would be quite unusable for me if its poor handling of image metadata could not be fixed in BB Pro.

Obviously, the purpose of the investigation was to find a way of improving application interoperability in one rather significant area. I strongly agree with that intent. However, I do have comments on the specific recommendations that have been made.


Clearly label the “Edit Comments” dialog box to reflect its state and purpose, i.e. EXIF Comments.
Agree that the terminology is unclear. Believe that the essential purpose of the Ctrl+E dialogue is not as stated, but to add a caption to an image. Where the caption is stored is a secondary issue, which only becomes relevant to most users when they are affected by interoperability issues.


Clearly label the User Comment / Description displayed in filmstrip and full views as to their source, i.e. EXIF, IPTC (legacy), or IPTC/XMP.
This could be useful. I'd rather see information in the Ctrl+E box as to where the caption is being saved, so that problems can be fixed at source.


Use the capability ... to create and populate an EXIF User Comment / Description field in an image file that contains no pre-existing EXIF User Comment / Description field, to enable population of the EXIF User Comment / Description field from the IPTC Description field on both a manual and @IPTC_description@ batch basis.
Agree that duplicating the data as proposed will often be useful (it would for me). But the implementation needs further thought; it is important that users have choice and are clear about what is happening. Also, I would write to legacy IIM data only if that option is selected. I do, however think that we have reached the stage at which captions should be written to both EXIF and IPTC/XMP by default, thereby giving new users the maximum chance of interoperability with other applications.

Enable manual and batch replication of all relevant IPTC fields to EXIF fields, i.e. owner/creator, description, copyright, etc.
This too would be useful, but also needs more thought. For example, as some cameras (particularly Canon) record an owner string in EXIF (which is, as a data set, intended to be mostly read only), there should be a default option not to overwrite fields that are already populated.

I hope that these comments are helpful. Again many thanks for the input, and in particular for putting the discussion on a much broader and more helpful basis than the more usual, "I have a problem, how do I fix it?".

[Chris Breeze]

Many EXIF fields are a minefield of muddled thinking and inconsistent use by different camera manufaturers. The standard defines tags for UserComment and ImageDescription. Some cameras write both, some only one and some neither. Some manufacturers store their name in the ImageDescription. Also the space allocated to these fields is sometimes set to only 16 characters.

The way BBPro works is it tries to use the UserComment field if present otherwise it tries the ImageDescription. If there isn't enough space for the comment BBPro does not attempt to rewrite the EXIF data to make more space as this could result in the MakerNote becoming corrupted (some apps don't worry about this and trash the MakerNote).

I recommend setting the option to "Use IPTC description/caption in preference to EXIF comments" as this will allow comments to be added to any JPEG without having to worry whether it already contains a suitable UserComment or ImageDescription field.

[DavidB]

Many EXIF fields are a minefield of muddled thinking and inconsistent use by different camera manufaturers. The standard defines tags for UserComment and ImageDescription. Some cameras write both, some only one and some neither. Some manufacturers store their name in the ImageDescription. Also the space allocated to these fields is sometimes set to only 16 characters.
This is useful information, for which I am grateful, but it does not invalidate the point that, whenever possible, BB Pro should write the data where other apps may look for it.


The way BBPro works is it tries to use the UserComment field if present otherwise it tries the ImageDescription. If there isn't enough space for the comment BBPro does not attempt to rewrite the EXIF data to make more space as this could result in the MakerNote becoming corrupted (some apps don't worry about this and trash the MakerNote).
Yes, I've had that, and it's not nice. But, there again, is it not possible to develop an algorithm (at least for the caption, which is probably the most used data item) to make it more likely that other apps will find it?


I recommend setting the option to "Use IPTC description/caption in preference to EXIF comments" as this will allow comments to be added to any JPEG without having to worry whether it already contains a suitable UserComment or ImageDescription field.
If that is the recommendation, should it not also be the default? (I've had it set for ages, so have forgotten what the default was on my system, but Help says it is not the default.) I reckon that change on its own would remove a number of the problems people have had.

[dhackney]

The work is important, in my view, because BB Pro is one of the few tools that helps users chart a path through through the standards minefield, particularly where metadata is concerned. With its level of standards compliance, BB Pro, when used as 'workflow central', actually enhances compatibility between applications.

I've used BBP and DP for a long time now. the goal of this current project was to eliminate a variety of utilities from my workflow and use BS products as much as possible, in fact, as the linchpins of the process.



Obviously, the purpose of the investigation was to find a way of improving application interoperability in one rather significant area.

That is correct. As users, we are forced to deal with the disaster the standards bodies, hardware manufacturers and software developers have made of meta data.

The specific issue I am attempting to resolve with BBP is significant and it will not be going away any time soon. There are more than a few applications and web services that are limited to EXIF data fields.


I hope that these comments are helpful.

Very helpful and I agree with your overall perspective on the issue.

The ideal solution would be a dialog and underlying capability such as this:
http://www.hackneys.com/photos/bbp-test/bpp-mappingdialog.jpg

The capabilities implied by the dialog box are all possible using the exiftools utility by Phil Harvey. It is completely capable of maintaining the necessary pointers to create and edit meta data fields in the various data types.

But, I don't know if it's possible to integrate that code into a commercial product.

The approach implied by the dialog box would enable free form mapping of meta data between XMP and EXIF fields. It scopes the number of displayed fields down to a manageble amount and it allows for control of overwriting target fields.

Valuable enhancements would be:

Radio buttons to select the direction of data flow, i.e. source and target on a field by field basis.
Radio buttons to select source and target meta data types, i.e. EXIF, IPTC and XMP (and others to follow).


It would be wonderful if meta data was a well behaved, extensible and professionally architected world, but unfortunately, it is much closer to anarchic chaos.

I don't see the challenge of managing image meta data getting any better. If anything, the vendors are continuing to fragment what semblence of standards exist, promising an even more chaotic future. I anticipate that some form of meta data mapping capability will become even more and more essential in the future.

[dhackney]

Many EXIF fields are a minefield of muddled thinking and inconsistent use by different camera manufaturers.

Very true, and probably a sizable understatement. My heart goes out to those of you who must navigate those troubled waters on a daily basis.

For those interested in learning about the foibles and failures of the various meta data standards here's an informative and sometimes humorous commentary on meta data by Phil Harvey, the developer of a usefull exif utility: http://www.hackneys.com/photos/bbp-test/commentary.html


If there isn't enough space for the comment BBPro does not attempt to rewrite the EXIF data to make more space as this could result in the MakerNote becoming corrupted (some apps don't worry about this and trash the MakerNote).

Another example of a poorly designed industry standard even more poorly implemented by the manufacturers. There are utilities such as Phil's exiftools that have overcome this significant challenge. I don't know if it is possible to integrate or license that code into a commercial product.



I recommend setting the option to "Use IPTC description/caption in preference to EXIF comments" as this will allow comments to be added to any JPEG without having to worry whether it already contains a suitable UserComment or ImageDescription field.

I agree this is a good default way of configuring and using BBP. However it does not overcome the fundamental issues that under certain scenarios BBP is not actually writing the displayed descriptions into the EXIF data set, applications and web services that rely on EXIF have no way to access the descriptions displayed in BBP and that BBP users have no idea where the descriptions they are looking at are coming from: EXIF, IPTC (legacy) or XMP.

Personally, I think it is a shame that devleopers like yourself are forced to clean up after the standards bodies, manufactures and mega-software companies. But, for customers such as us, you are our only hope for powerful, useful tools to overcome these real-world challenges.

[dhackney]

Here's a workaround for the BBP XMP/EXIF meta data field bug.

It utilizes a free windows command line exif utility. For those of you on UNIX or MAC environments, there is a PERL version of the utility that runs on just about anything.

Requirements:

1. An unzipped copy of the Windows command line exiftools utility by Phil Harvey available here: http://www.sno.phy.queensu.ca/~phil/exiftool/

Download the windows executable and the Perl library version. You will eventually want the HTML documentation files in the Perl version.

2. The batch file and parameter files here: http://www.hackneys.com/photos/bbp-test/

a. BBP-xmp2exif-usingargsfile.bat (http://www.hackneys.com/photos/bbp-test/BBP-xmp2exif-usingargsfile.bat)
b. bbp-xmp2exif.args (http://www.hackneys.com/photos/bbp-test/bbp-xmp2exif.args)

Configuration:

1. Unzip the exiftool utility.
2. Rename the utility exiftool.exe
3. Copy the exiftool.exe file to your windows folder or on your system path
4. Copy the batch file and parameter file to the folder with the files you wish to process

Utilization:

1. In BBP, enable preference "Use IPTC description/caption..."
2. Use BBP "Edit:Edit XMP/IPTC Data" dialogs to populate image meta data.
3. Open Windows Explorer
4. Navigate to the folder where you placed the batch file and parameter file containing the JPG files you wish to process
5. Double click on the batch file BBP-xmp2exif-usingargsfile.bat
6. All relevent XMP/IPTC meta data created by BBP is copied into the corresponding EXIF meta data fields and will be available to applications and web services that are limited to or require EXIF data.

Notes:

1. This batch file will overwrite the original JPG files with the new EXIF data fields populated. It is recommended you have a backup copy of your JPG files.
2. The parameter file makes the following associations. The XMP meta data fields are written to the corresponding EXIF fields. If no correspondiing EXIF field exists, it is created and all meta data file internal pointers are updated accordingly.


image title to EXIF
-XMP-dc:Title > EXIF:ImageDescription
image description to EXIF
-XMP-dc:Description > EXIF:UserComment
image description to JPEG comment
-XMP-dc:Description > Comment
image creator to EXIF
-XMP-dc:Creator > EXIF:Artist
copyright statement to EXIF
-XMP-dc:Rights > EXIF:Copyright

If you desire or require additional EXIF fields to be populated you can edit the parameter file.

The parameter file can also be edited to enable population of IPTC fields from XMP fields. This is useful if you did not have the "Also store IPTC data in legacy..." preference enabled and desire to populate your files' IPTC data fields. Note that some applications do not support XMP and require the IPTC fields to be explicitly populated.

[dhackney]

The batch and parameter files are documented internally. Open with any text editor.

To learn more about the exiftool commands and options used, refer to the HTML documentation files included with the Perl utility set.

The relevent XMP/IPTC/EXIF/JPEG meta data fields used in the workaround are documented here: http://www.hackneys.com/photos/bbp-test/bbp-xmp2exifmetadatafields.pdf

[dhackney]

Here are some useful batch files I put together while building the workaround. They display the available meta data fields in a jpg file.

All require the same prerequisites as the workaround.

If you want to map different or additional EXIF, IPTC or XMP meta data fields you will first need to learn what your camera uses as field (tag) names for the values you are interested in.

In order to do that I created three batch files to create three different text files of meta data information: EXIF, XMP and All.

Once these files are created for files from any given camera you will have the information you need to modify the parameter file to map any source field to any target field.

All of these batch files use an image file named sample.jpg. I provided a sample.jpg file here to get you started: http://www.hackneys.com/photos/bbp-test/sample.jpg

All batch files create corresponding txt files of their output that can be opened in any text editor.

Notes:
1. You must eliminate any spaces in tag names, i.e. User Comment needs to be UserComment for the mapping to work.
2. An XL spreadsheet that automates the creation of the code strings for use in the parameter file is here: http://www.hackneys.com/photos/bbp-test/bbp-xmp2exifmetadatafields.xls

Batch Files:
1. list-exif-tags-sample.bat

Batch to display all available EXIF meta data fields (tags) for a jpg file.
Available here: http://www.hackneys.com/photos/bbp-test/list-exif-tags-sample.bat

To use:
a. create a copy of a jpg you are interested in, i.e. for a specific camera
b. rename the copy sample.jpg
c. put the list-exif-tags-sample.bat in the same folder as the jpg
d. open windows explorer
e. double click on list-exif-tags-sample.bat

A sample of the output of this batch file is here: http://www.hackneys.com/photos/bbp-test/sample-exiflist.txt


2. list-XMP-tags-sample.bat

Batch to display all available XMP meta data fields (tags) for a jpg file.
Available here: http://www.hackneys.com/photos/bbp-test/list-xmp-tags-sample.bat

To use:
a. create a copy of a jpg you are interested in, i.e. for a specific camera
b. rename the copy sample.jpg
c. put the list-xmp-tags-sample.bat in the same folder as the jpg
d. open windows explorer
e. double click on list-xmp-tags-sample.bat

A sample of the output of this batch file is here: http://www.hackneys.com/photos/bbp-test/sample-xmplist.txt


3. list-metadata-tags-sample.bat

Batch to display all available meta data fields (tags) for a jpg file.
Available here: http://www.hackneys.com/photos/bbp-test/list-metadata-tags-sample.bat

To use:
a. create a copy of a jpg you are interested in, i.e. for a specific camera
b. rename the copy sample.jpg
c. put the list-metadata-tags-sample.bat in the same folder as the jpg
d. open windows explorer
e. double click on list-metadata-tags-sample.bat

A sample of the output of this batch file is here: http://www.hackneys.com/photos/bbp-test/sample-metadatalist.txt

[MikeD]

I normally fill in IPTC fields with BBpro as a first step then add lat long information, this works fine with most types of file, however with .cr2 files Robogeo wipes out all the IPTC information when you try to add the georeference data.
Robogeo claims not to wipe any information but each time I have tried it does, using Robogeo version 5.3
The IPTC data is not entered directly into the .cr2 file but in a .xmp sidecar file so it seems rather strange that these should be damaged. I assume this effect is something to do with the standards issues mentioned above but so far not found a work around other than trying to use Robogeo first before any IPTC info is added. However this does not help for the thousands of files where I already have IPTC in but not yet put the geocode data.

[DavidB]

Perhaps this is indeed another example of the IPTC updating problems which seem to plague almost every application. Some applications simply do not acknowledge a pre-existing XMP file and over-write it with a new one

As far as I know, BB Pro conforms fully to the IPTC/XMP Core Schema. An oddity is that some of the values are actually recorded in the Photoshop Schema (because that is the way Adobe set up the IPTC/XMP standard), but you are right in thinking that this should not affect the way that Robogeo writes to the XMP file.

As XMP files are in plain text, and are easily read in Notepad or any other text editor, it is worth doing a few tests to see what actually happens to the file in RoboGeo. Also, if you write the data to the EXIF tags rather than the IPTC, does this affect the XMP file? Do you have the same problems with a JPEG conversion (including full EXIF/IPTC data of course) of the original RAW.

At the moment, this sounds more like a RoboGeo problem than a BB Pro problem, but, if your investigations indicate otherwise, you can always send a file or files to Chris for examination.

Hope this helps.

[dhackney]

I normally fill in IPTC fields with BBpro as a first step then add lat long information, this works fine with most types of file, however with .cr2 files Robogeo wipes out all the IPTC information when you try to add the georeference data.
Robogeo claims not to wipe any information but each time I have tried it does, using Robogeo version 5.3
The IPTC data is not entered directly into the .cr2 file but in a .xmp sidecar file so it seems rather strange that these should be damaged. I assume this effect is something to do with the standards issues mentioned above but so far not found a work around other than trying to use Robogeo first before any IPTC info is added. However this does not help for the thousands of files where I already have IPTC in but not yet put the geocode data.

I am using RoboGEO 5.3.9.

I am also experiencing the same data loss of XMP-IPTC data after processing files with RoboGEO. I have not done any testing on this issue. I have not identified any work-arounds.

I use Breeze DownLoader Pro (DLP) to lay in Geocode data on download, but DLP does not have "nearest track point" capability. Consequently, I often need to use RoboGEO to add geocoding info to images that are not geocoded by DLP.

I put RoboGEO immediately after DLP in my workflow to limit destruction.

But, like you, that doesn't help with images that are already carrying a load of XMP-IPTC data that need geocode data.

Aside from RoboGEO getting fixed, perhaps the best way would be to create copies of all such images in another folder, use RoboGEO to lay in the geocode data, then use an exiftools batch to copy the XMP-IPTC and other meta data from the originals to the newly geocoded copies.

1. Create folder of image copies (imagecopies)
2. Using RoboGEO, lay in geocode data for files in imagecopies folder
3. Using exiftools batch, copy all non geocode meta data from image originals to files in imagecopies folder

Please note that I'm not an exiftools expert, so I cannot offer any examples of how to write that batch.

But, I've got a bunch of RoboGEO ravaged images to repair, so I guess I'll need to do so! :)

[dhackney]

Do you have the same problems with a JPEG conversion (including full EXIF/IPTC data of course) of the original RAW.

On my system and in my workflow, JPEG conversions and sidecar files are unaffected by RoboGEO. It appears to be limited to raw files' XMP sidecar files.

Please note I have not done a meta data audit on this issue.



At the moment, this sounds more like a RoboGeo problem than a BB Pro problem

Agreed.

But, it sure would help if DLP had "nearest track point" capability, which would elimiate most of my need to use RoboGEO. :)

[dhackney]

Update on RoboGEO.

My first quick check of the RoboGEO issue led me to believe I was experiencing the same issue.

This initial quick check was not correct.

I did a more complete quick check and was unable to duplicate the issue.

At this time I cannot duplicate the reported issue with RoboGEO corrupting or erasing the contents of XMP sidecar files.

Again, at this time, I have not done a full meta data workflow audit of this issue, but a more thorough check shows that all meta data in the XMP file survives a RoboGEO "Write the location into the EXIF headers" operation.

[Chris Breeze]

But, it sure would help if DLP had "nearest track point" capability, which would elimiate most of my need to use RoboGEO. :)

Please can you explain what you mean by "nearest track point capability"?

Downloader Pro reads the GPS track log and finds the nearest track point before and after the time the photo was taken and interpolates between the two to calculate the position. Since a track log may span days or weeks, DLPro limits the time window for comparing fixes (no data is usually preferable to old, incorrect data). This value defaults to 5 minutes (i.e. comparing the nearest fix within +/- 5 minutes of the picture being taken) but can be set in the GPS setting window. If you open it up to several hours it won't affect the accuracy of the data when there are GPS fixes close to the time when the photo was taken but will allow older fixes to be used if necessary.

[dhackney]

Chris,

Thanks for posting on this.

RoboGEO has a preference setting for using the closest track point:

http://www.hackneys.com/photos/robogeo-closest-track-point.jpg


In terms of real-world utilization, we shoot travel photography, so there are times and places when we will be in the same basic location for days or even weeks.

Because we travel overland, we always have a tracklog for our journey into and out of a shot location. However, if we are shooting a festival or a market over a series of days, we are very unlikely to carry a portable GPS with us during those shoots. The RoboGEO flag allows us to embed our base location onto each image from that shoot.

Because we also shoot during our overland travel and while trekking, etc. carrying a portable GPS, I try to maintain reasonable tracklog accuracy when using DLP, so I keep the DLP time window at the default.

A single DLP download session often contains images from both of these scenarios, so changing the DLP acceptance window based on image type is not practical in day-to-day use.

I currently use RoboGEO to lay in the geocode data for all the images that fall outside the DLP acceptance window.

I don't know if our scenario is typical for your installed base and prospective customers or not, but perhaps a "Use Closest Track Point" flag could make the list of enhancements for a future version. That would allow DLP users to avoid using, buying, debugging, maintaining and upgrading yet another tool.

Fewer tools and a simplified workflow is a very good thing. Especially out here in the field... :)

Doug

[Clive]

Hi, Doug.

I am jealous of your activities that give rise to this problem. I am also puzzled.

If you ensure that you record a GPS location at the base station, say every 24 hours. and set the "nearest point" definition of "nearest" to say 18 hours, this will not affect DLP handling of on-location tracking, but will pin non-tracked excursions to your base.

Why is this not what you want?

[Chris Breeze]

I'm not sure there's much diffence between RoboGeo's "Use nearest trackpoint" and setting DLPro's GPS fix window to a large value like two days. DLPro will always use the nearest fixes before and after the time the photo was taken and interpolate between them and so the accuracy won't be affected when accurate GPS data is available.

[dhackney]

DLPro will always use the nearest fixes before and after the time the photo was taken and interpolate between them and so the accuracy won't be affected when accurate GPS data is available.

Chris,

This is the key information. Thank you for clarifying this point. I agree with your assessment that given this behavior of DLP there would be no functional difference between the two products.

I went back and re-read the documentation on the "fix window" in the help file:
The "GPS fix window" setting specifies how close the time of a GPS fix in the track log needs to be to the time the picture was taken for it to be accepted. Setting a large value will increase the chances that a GPS fix will be found in the track log but it may not be very accurate if the GPS track log has gaps in its data e.g. if you turn the GPS on, take some pictures in one location for a while then turn the GPS off, drive to a new location and forget to turn the GPS on again before taking more pictures DLPro will record the first location for the second set of pictures if the "GPS fix window" is set to a large value. If the "GPS fix window" is set to a small value (smaller than the time to drive from the first location to the second location) it won't find a suitable GPS fix in the track log.

Please note: Setting a large value for the "GPS fix window" only affects the accuracy of the GPS data stored in the image if there are gaps in the GPS track log. If the GPS track log contains a fix within a few seconds of when the picture was taken an accurate position will be stored in the shooting data of the image even if the "GPS fix window" is set to a large value.

In hindsight, I believe you do a good job of explaining the behavior there, especially with the emphasis on
If the GPS track log contains a fix within a few seconds of when the picture was taken an accurate position will be stored in the shooting data of the image even if the "GPS fix window" is set to a large value.

In setting up my workflow, I believe I must have been unsure about the warning
Setting a large value will increase the chances that a GPS fix will be found in the track log but it may not be very accurate if the GPS track log has gaps in its data e.g. if you turn the GPS on, take some pictures in one location for a while then turn the GPS off, drive to a new location and forget to turn the GPS on again before taking more pictures DLPro will record the first location for the second set of pictures if the "GPS fix window" is set to a large value.

This behavior is actually exactly what we reguire, but I did not perceive that when I read the help file when creating the workflow.

In some scnearios such as ours, admittidly rare, the large value actually is an asset, and enables a capability, rather than a potential negative. I'm not sure how to best express that in the help file other than describing the scenario.

Thanks again for clarifying DLP's behavior in this regard.

Doug

[dhackney]

I am jealous of your activities that give rise to this problem.

It's not all glamour...

We're currently caught in a very rare cold wave in south-central Chile. We've been trading off standing pump watch all night. We have to manually cycle one of our water pumps every hour to keep those water transfer lines from freezing.

I wasn't worthy of much jealousy last night at 3AM, bleary-eyed, counting the seconds off that hour's pump cycle. :o




I am also puzzled.

If you ensure that you record a GPS location at the base station, say every 24 hours. and set the "nearest point" definition of "nearest" to say 18 hours, this will not affect DLP handling of on-location tracking, but will pin non-tracked excursions to your base.

Why is this not what you want?

Clive,

See previous reply to Chris. I agree that this will provide the behavior we seek. I believe now, with 20/20 hindsight, that I misunderstood, or did not fully understand, DLP's behavior related to the "fix window" as described in the help file.

Thanks for pointing out that capability.

Doug

[ggivensjr]

Hi all,
Ok, your discussion is way over my head but I have a question about the EXIF/IPTC data I add to a file in BBPro. When I open the image in PSPro the information I added in BBPro is not there. Why is it for example the following information does not show in PSPro:
Document title: Alexann and Vicki (Image Title in PSPro)
Description: Picture of Alexann and her aunt Vicki at high school graduation open house. (Same in PSPro)
Description writer: Photographer (Same in PSPro and is written)
Author: George E. Givens; Jr. (Artist* in PSPro)
Author title: Photographer
Intellectual genre: From the Scene
Keywords: alexann gieseking, graduation, 062008, high school, vicki, education
IPTC subject codes: 05005003; 08003003
IPTC scene: 011800; 012100; 010800

Thanks,
George

[DavidB]

Ok, your discussion is way over my head but I have a question about the EXIF/IPTC data I add to a file in BBPro. When I open the image in PSPro the information I added in BBPro is not there.
You don't say which version of PSP you are using, and it's been through so many different versions and changes of ownership that it's a bit difficult to be specific. But there is one possible cause that seems most likely.

Firstly, the information is almost certainly there. You can check that by opening the image in BB Pro, after reading but not saving it in PSP. The problem is most likely that PSP (at least in your version) can't read the data, and will not write it when it saves the file.

IPTC data comes in 2 flavours. The older standard (properly IPTC/IIM) is sometimes, confusingly, called just 'IPTC'. The newer standard (IPTC/XMP) is sometimes referred to as just 'XMP'; that too is confusing, because XMP data, when present, usually contains much more than the IPTC data. The 2 standards store data in different ways; older applications, and many current ones, still support only IIM; all Adobe applications, however, are XMP-only (Adobe invented XMP).

Breeze applications only write XMP data by default, but will read and can be configured to write 'legacy' IIM data as well. In BB Pro, the setting is on the General tab of preferences. Try enabling this setting, write some IPTC data to an image, and see if PSP will recognise it.

If this works, that is only half your problem solved, because PSP will trash the XMP data when it saves the file. The answer is always to save the images you work on in PSP as copies (a good practice in any case); BB Pro has facilities for copying metadata from a number if file formats to JPEGs and (with some limitations) TIFFs; you can read about these in Help if you need to use them.

This is very much a simplified answer; metadata is a complicated subject (aka a mess), because of a combination of ignorance, stupidity and pursuit of self-interest by standards setters, camera makers, software writers and users alike.

Hope this helps.

[dhackney]

George,

I concur with the "mess" description of the state of meta data.

The only way to know, with certainty, what is happening to your photo's meta data in your workflow, using your tools, is to do a step by step, tool by tool, audit of the meta data across your workflow.

This will take from 10 minutes to an hour, depending on how complex your workflow is and what tools you use.

There are two parts to this process:
A. Process a test photo or photos across your workflow while saving a version of the photo after each step.
B. Audit the meta data of the saved photos by viewing the meta data from each file.


A. Processing the Test Photo(s)

1. Create a folder on your computer named "meta data audit." Put all subsequent folders from the test inside this one.

2. Inside the "meta data audit folder" create a folder named "00 - originals."

3. Inside the "meta data audit folder" create a series of folders for each step in your workflow, such as “01 – DLP,” "02 - BBP," "03 - PSP," "04- Noise Ninja," etc. It will help you later if you name the folders with leading numbers. That ensures the folders will sort and display in the order of your workflow.

4. Using Windows Explorer, copy a fresh, untouched, unprocessed test photo created by every camera you use and place it into the folder named "00 - originals." Note that in this step you are NOT using your normal photo file downloader tool, you are directly copying the camera’s photo file from the memory card to the computer, with no processing, no added meta data, etc.

5. Process your test photo(s) through each one of the tools in your workflow. Be sure to save them into the folder for that tool AFTER that tool's step.

6. If your first step is to download the photos in Breeze Downloader Pro (DLP), then download the images from your memory card(s) using DLP and configure DLP to put them into your "01 - DLP" folder. If you use a manual process or another tool for downloading, use that process or tool to download the photos into your “01 – post download” folder.

7. Next, perform the subsequent steps in your workflow and save the test photo(s) into that tool's folder. For instance, if your next step is to use Breeze Browser Pro to perform the "Ins and Outs" culling and initial IPTC meta data population, then using Windows Explorer or BBP, copy the photo from "01 - DLP" into "02 - BBP" and then do a sample IPTC meta data addition. Note that in this step you MUST copy the photo into the “02 – BBP” folder BEFORE you do anything with the photo’s meta data in BBP.

8. In your case, the next step may be some cropping in PSP. If so, select the test photo in BBP and using CTRL D or Edit:Edit Image, open the file in PSP (this assumes you have PSP assigned as your photo editor in BBP, if you do not, then open the photo directly in PSP). Using PSP, add some additional meta data and, using the “File:Save As” command, save the photo in the "03 - PSP" folder. (Tip: It can help later if you add the step or tool information to the photo filename after you perform each step of the workflow. For example, after the PSP step you would save the test photo file as TESTPHOTO-step-03-psp.jpg.)

9. Repeat this process with all tools used in your workflow.


B. Auditing the Meta Data.

1. Create a spreadsheet or a document with a grid listing all the meta data fields you are interested in down the left, vertical axis and all the tools you use in your workflow across the top, horizontal axis. If you use MS Word, use the Table:Insert Table menu option.

http://www.hackneys.com/photos/metadataauditsample01.jpg




2. Using your meta data audit tool(s), check for the existence and accuracy of the meta data fields you are auditing in each of the photos all the test folders. For this audit, at a minimum, include your primary workflow tool, such as BBP.

Note, however, that even BBP has some quirks regarding meta data. It too, may not display all the meta data accurately or clearly label the source of the meta data (description/caption information). For this reason, it is best to use a few different tools to audit the meta data you are interested in.

There are a variety of meta data viewing tools mentioned in the first posts on this thread. You will need to understand some of the basics about meta data to properly interpret their results.

For instance, older tools such as Exifer and Paint Shop Pro 9, were developed prior to XMP, so they have no understanding of it or ability to read or display it. On the other hand, Exifer can read and display legacy IPTC/IIM meta data, so it is a great way to see if a tool such as BBP is actually writing the meta data to the IPTC/IIM area.

As you’ve noticed, different tools use different label names for the same piece of meta data. And, as pointed out in the previous posts, different tools display different meta data stored in different places, even if they call it the same thing, such as “description” or “caption.” This means you will need to populate the meta data with descriptive text, such as “Image description text,” “artist name text,” “copyright statement text,” etc. Using descriptive text is often the only way to understand that different tools are using different labels for the same piece of meta data.

The results of your meta data audit will show you if all of the meta data you are interested in is surviving your work flow. If not, it will show you exactly where the meta data is being lost or corrupted so you can modify your workflow accordingly to restore or correct it.


ExifTool note:
The only method I am aware of to achieve a full, complete and accurate dump of a photo’s meta data is using the windows command line ExifTool by Phil Harvey method. It’s a little bit technical but it will give you all the meta data in its actual and complete form. Note that the ExifToolGUI utility is much easier to use but it does not display the full set of meta data. However, it may be adequate for your uses.

Adobe note:
Even if, like me, you have spent many years avoiding the photo tools sold by Adobe, you should probably use one of their current version tools to take a look at your meta data. It is important to know what those tools are displaying, especially when sharing your photos with others. Running a test with the Adobe tools will show you what other people using Adobe tools will see when they look at your photos’ meta data. Please keep in mind that there are differences in meta data display, labeling, sourcing, etc. across the Adobe product range.

[technical notes follow]

Doug

[dhackney]

Technical note:
Winston Churchill’s immortal description of 1939 Russia as “A riddle wrapped in a mystery inside an enigma” is equally true of the current state of digital photography meta data.

Meta data can be a very technical, daunting subject. In practice and application in digital photography, it has often been an unmitigated disaster. In an effort to shield users from this gory mess, software developers often use nicely formatted displays and comforting, familiar names, such as “Description” or “Date,” when displaying meta data.

Unfortunately, the rule of unintended consequences comes into play and this well meaning effort to protect the users from the underlying complexity has often led to misunderstandings, lost and corrupted meta data, and widespread user community confusion.

Most of the confusion relates to several primary factors:
1. There are multiple types or classes of meta data. The relevant classes include EXIF, IPTC-IIM, and XMP.

The conceptual model of digital photography meta data may be considered as:

http://www.hackneys.com/photos/meta-data-conceptual-model-02-crop.jpg

EXIF (Exchangeable Image File Format) meta data is initially populated by the digital camera.
EXIF information is formatted according to the TIFF specification, and may be found in JPG, TIFF, PNG, MIFF and HDP images, as well as many TIFF-based RAW images, and even some AVI and MOV videos. EXIF Maker Notes meta data contains proprietary camera manufacturer information and varies from camera to camera, even from the same manufacturer and within the same product family.

IPTC-IIM (legacy) (International Press Telecommunications Council) meta data is an older standard that is generally being phased out in favor of XMP. IPTC information may be embedded in JPG, TIFF, PNG, MIFF, PS, PDF, PSD and DNG images.

XMP (Extensible Metadata Platform) is a standard created by Adobe. XMP is an XML/RDF-based metadata format. It can be embedded in many different image file types including JPG, JP2, TIFF, PS, PDF, PSD, DNG, PNG, SVG and MIFF, as well as audio file formats supporting ID3v2 information. XMP is an extensible meta data structure, meaning different tools can create and utilize meta data within the XMP data structure. XMP is segmented into multiple sub-classes called namespaces. XMP field (tag) names can be duplicated between namespaces. A full XMP field (tag) name includes the namespace, e.g., XMP-exif:Contrast or XMP-crs:Contrast. There are namespaces within XMP that are named and/or contain EXIF and IPTC data.

Digital photography meta data is contained within JPEG files. For RAW files, XMP data is written to a separate file, usually called a “sidecar” file. The sidecar file is a text format file and usually shares the RAW photo file name with an .xmp file type extension, e.g., IMG_1234.xmp.

Each of the three main classes of digital photography meta data: EXIF, IPTC-IIM and XMP exist as independent entities. None of them inherently knows about the existence of or the contents of the other classes. It is up to each tool that reads, displays, edits and writes meta data to correctly display the meta data, keep it in synch with each other, and correctly, in a non-destructive way, write the meta data.

It is all too common for tools to corrupt or destroy existing meta data when they write to the photo file, such as when editing a photo and saving it, or when creating, adding to, editing, and subsequently writing to the photo’s meta data.

There are no international standards related to how tools must, or even should, display or label meta data. It is up to each tool’s developers to decide how they will label and display the meta data. For instance, a tool can display or label the field that contains the date/time the image was created as “Date” or “Creation Date” or “Image Date.” The same is true for every single element of meta data, which is why “description” or “caption” can appear under so many different names or labels in so many different tools.

In addition, there is widespread duplication of photo meta data across the three classes, especially between EXIF and XMP. Meta data such as date/time and description/caption exist in both classes, and can be independently edited by various tools.


2. The EXIF meta data class was the initial digital photography meta data class. Cameras embed EXIF metadata such as shutter speed, f stop, etc. into photo files as they are created.

XMP was not and is not part of the EXIF standard. If a tool was developed prior to XMP, such as Exifer or Paint Shop Pro (PSP) 9, then that tool has no way to view or edit XMP metadata.

Adobe, Breeze Systems, and a growing number of other developers and products support XMP metadata.

If a tool only displays, edits and maintains XMP, then the two data sets, EXIF and XMP, can become de-coupled for common data points that exist in both classes, such as date/time.

If a tool, such as RoboGEO, relies on the EXIF version of a common data point but that common data point, such as date/time, was edited by a tool that only maintains XMP data, such as Adobe Lightroom (LR), then the EXIF dependent tool will be working with out-of-synch data.


3. The XMP specification has a namespace area labeled EXIF. Adding to the confusion even more, the XMP-EXIF namespace has field names that exactly duplicate those of EXIF field names.

This means that a tool can display an XMP-EXIF namespace class meta data field using the label “EXIF Date.” Unfortunately, that XMP-EXIF class field may contain data that is different from the EXIF class date/time field.

[editorial comment] The developers of the tool probably consider themselves correct in labeling the field “EXIF Date” since the data comes from the XMP-EXIF namespace date/time meta data field. They might even consider themselves bold, forward thinking standard setters for completely ignoring the old, and in their eyes, completely outdated, EXIF meta data date/time field. But, however stylish and trendy they might be in the developer community, their actions create a user community that is betrayed, and in the end, confused and frustrated.

No matter what the developers would like the world to be like, the real world is still populated by a wide variety of tools, many of which rely on, primarily or exclusively, EXIF meta data. It is up to the developers and their tools to maintain consistency across all common meta data in all meta data classes.

For instance, if a tool writes to the XMP-dc:Description field, it needs to also write to the EXIF ImageDescription field and the JPEG Comment field. [end editorial comment]


4. BBP, LR, Photoshop Elements (PSE), Adobe Bridge, etc. use, either exclusively or primarily, XMP metadata, even though they may label/display it as EXIF. LR, for instance, only displays the XMP date/time values, even though they are labeled EXIF.


5. The EXIF specification assigns field names different than those commonly used in various tools when labeling/displaying EXIF metadata fields. For instance, you will find the very same EXIF date/time field labeled a wide variety of different ways in a sample set of even a handful of different tools.


Example

A useful example of the widespread tool disparity, data duplication, and naming duplication can be found in Date/Time meta data.

Relevant Time Metadata Fields

EXIF Metadata

Tag ID…….Tag-Name…………………….Comments
-----------------------------------------------------------------------------------------------------------------------------
0x0132…… ModifyDate…………………. (called DateTime by the EXIF spec)
0x882a…… TimeZoneOffset…………….. (1 or 2 values: 1. The time zone offset of DateTimeOriginal from GMT in hours, 2. If present, the time zone offset of ModifyDate)
0x9003……. DateTimeOriginal
0x9004……. CreateDate………………….. (called DateTimeDigitized by the EXIF spec)


XMP-EXIF Namespace Metadata
(XMP metadata is segmented by namespaces. The main section is called Dublin Core, commonly referred to as dc. XMP also has a namespace called EXIF.)

Namespace……..Tag-Name……………….Comments
-----------------------------------------------------------------------------------------------------------------------------
dc……………….Date
EXIF……………DateTimeDigitized………[note duplication of original EXIF specification field name]
EXIF……………DateTimeOriginal………..[note duplication of EXIF field name]

(source: exiftool by Phil Harvey documentation, [ ] comments are my own)

Note that in LR, the date/time information displayed in the Metadata panel is not the EXIF date/time field metadata, it is the current contents of the XMP-EXIF namespace date/time field metadata contained in the LR database.



Sources:
- ExifTool by Phil Harvey documentation


Terminology note:
I used the database term “field” to denote meta data “tags.” My goal was to avoid confusion between meta data tags and the term “tags” used in tools such as PSP and PSE for keywords and categories.

[ggivensjr]

You don't say which version of PSP you are using, and it's been through so many different versions and changes of ownership that it's a bit difficult to be specific. But there is one possible cause that seems most likely.
I have PSP X and XI. The data from this example came from X. I am still using W2K on that computer because XI isn't compatible with W2K.


Firstly, the information is almost certainly there. You can check that by opening the image in BB Pro, after reading but not saving it in PSP. The problem is most likely that PSP (at least in your version) can't read the data, and will not write it when it saves the file.
You are correct. I did not resave the file in PSP and when I reopened it in BBP the data is intact.


IPTC data comes in 2 flavours. The older standard (properly IPTC/IIM) is sometimes, confusingly, called just 'IPTC'. The newer standard (IPTC/XMP) is sometimes referred to as just 'XMP'; that too is confusing, because XMP data, when present, usually contains much more than the IPTC data. The 2 standards store data in different ways; older applications, and many current ones, still support only IIM; all Adobe applications, however, are XMP-only (Adobe invented XMP).
I forgot about that little fact :o. I don't understand why when the XMP standard was created it was not created to extract the the older IPTC/IIM and newer IPTC/XMP if present and convert it to the correct fields for XMP. Am I over simplifying?


Breeze applications only write XMP data by default, but will read and can be configured to write 'legacy' IIM data as well. In BB Pro, the setting is on the General tab of preferences. Try enabling this setting, write some IPTC data to an image, and see if PSP will recognise it.
Will BBP write both the XMP and legacy IPTC?


If this works, that is only half your problem solved, because PSP will trash the XMP data when it saves the file. The answer is always to save the images you work on in PSP as copies (a good practice in any case); BB Pro has facilities for copying metadata from a number if file formats to JPEGs and (with some limitations) TIFFs; you can read about these in Help if you need to use them.
Since, when I edit files and resave I always "save as" and rename (sometimes to the same folder) I can get around this by doing as Doug has suggested in his meta data audit test and create a seperate folder inside the original named PSP and rewrite the data in the PSP version that didn't come over from BBP version. This is a hassle I know but since it is on a file by file basis and since some of the data is coming over to PSP I don't think it will be that big of a sticking point.


This is very much a simplified answer; metadata is a complicated subject (aka a mess), because of a combination of ignorance, stupidity and pursuit of self-interest by standards setters, camera makers, software writers and users alike.

Hope this helps.
Are you kidding? You have been a really big help! Thank you very much for taking the time to respond.

[ggivensjr]

Doug,

Thank you for your well written explanation of the meta data issue and for your suggestion to do a meta data audit. I will get started on it as soon as time permits. I have to shoot at the childrens fishing derby this weekend and then get ready for the Indy Jazz Fest this coming weekend. But since my workflow is pretty much exactly as you described there will be no problem doing the audit.

Your insights have solved a mystery for me. I have been wondering why when I upload a file to Flickr that was worked in BBP then edited and saved in PSP the file did not retain the same data as a file worked in BBP only and uploaded to Flickr. Now I know, thank you!

Doug, do you mind if I make a copy of your posts to save for my own use as a tutorial?

Thank you for taking the time to respond. Are you a teacher? If you are you must a great one and if not you should have been a teacher.

Best Regards,
George

[dhackney]

I have PSP X and XI.

I just ran a quick test with PSP X2. Emphasis here on "quick."

Observations:
1. PSP X2 does not appear to be corrupting or deleting any of the IPTC data populated by BBP.
2. The BBP "description" and the PSP X2 "description" are two different pieces of meta data. The BBP "description" is the XMP-IPTC "Caption-Abstract" field (tag) and the PSP X2 "description" is the JPG "comment" field. Niether BBP or PSP X2 populate the EXIF "user comment" field. This can affect whether your "description" data appears on your chosen photo sharing web site.



I don't understand why when the XMP standard was created it was not created to extract the the older IPTC/IIM and newer IPTC/XMP if present and convert it to the correct fields for XMP. Am I over simplifying?

There you go, thinking like a software product user again... :)

No, you are not over simplifying from a customer's perspective. A customer expects the technology to work. A customer expects that if the software industry designs and develops a new standard that it will integrate with, and carry forward, previous standards. A customer expects that new standards will be backward compatible with the customer's existing collection of thousands of images and suite of workflow tools.

In these regards, the customer is usually disappointed.

There are three factors and entities at work here:
1. Standards forming teams/companies/organizations
2. Standards certification organizations
3. Software companies and their development teams

Anybody, even you and I, can proclaim a new standard. We can tout our new standard within our industry, champion its cause from the podium at trade shows and pump money into PR to gain its endorsement from the trade press.

To reach critical mass of market acceptance via this approach requires a lot of time and a lot of money. I don't know if George-Doug, Inc. would have those resources. A company like Adobe, however, has plenty of money to spend, so they can afford to push any standard they like.

If we felt our new standard had achieved a critical mass level of market acceptance, we could submit our new standard to an international standards body and seek to achieve status as a recognized and accepted international standard. For example, Adobe is currently seeking this for DNG.

But, even if we dominate our industry and achieve international standard status, it is still up to the individual software companies and their development teams as to how that standard will be implemented and supported.

Software development teams, by their very nature, are forward oriented. It is not a rewarding or promising career path for a developer to be assigned to a rearward view, such as documenting the code or building backward compatability with previous standards.

Developers want to be working on the latest, greatest, most advanced things. It gives them status within their community and keeps their resume fresh and marketable.

It is for these reasons that tools like Lightroom always have the latest, greatest, RAW processing capability contrasted by simple bugs that destroy legacy meta data.

Does that make sense from a customer/user perspective? No, it doesn't.

As users, we expect things to work. We expect industries to maintain backwards compatibility with their previous standards. But, in this case, our expectations run against the grain of the realities of the software marektplace.




Will BBP write both the XMP and legacy IPTC?

Yes.

1. Open BBP
2. Select File:Preferences
3. Enable the option "Also store IPTC data in legacy IPTC IIM format"



Doug, do you mind if I make a copy of your posts to save for my own use as a tutorial?

No, not at all.

I am working on compiling them into a coherent document that I will add to the "How To" section of our travel web site: http://www.hackneys.com/travel/. I hope to have it posted in the next few days.


Thank you for taking the time to respond. I am glad I was able to help.


Are you a teacher?
Not formally. I spent some time as a software instructor and then spent many years as a technology, marketing and management consultant to high technology companies.

[Clive]

I don't understand why when the XMP standard was created it was not created to extract the the older IPTC/IIM and newer IPTC/XMP if present and convert it to the correct fields for XMP. Am I over simplifying?

I think it is important to remember that the P in IPTC is "Press". Although more sectors are now involved, the standard still has its origins in daily publishing. "What does the editor/sound desk require when an image arrives via Packet-Switched Network, or more recently Internet, to get the story out in the next 12 hours?"

Such images have a very short life - half will be binned within 24 hours; a few more will be archived. "Oh, you want to use the standard for your purposes? Fine, as long as you don't bleat about how the standard does not fit your personal expectations."

At the other end is the Archivist community, with most photographers wanting a little of each approach. Will a unified approach be developed? Hmmm.

[DavidB]

George

Doug and now Clive have pretty much responded to all the issues that you (usefully) raised. Like you, I am in awe of Doug's rigorous approach to these issues, and I have taken his permission as agreement that I can save his really useful conceptual model for future reference.

I would like to add my own thoughts to Doug's answer to your question


I don't understand why when the XMP standard was created it was not created to extract the the older IPTC/IIM and newer IPTC/XMP if present and convert it to the correct fields for XMP. Am I over simplifying?
I agree 100% with Doug's answer. At the time and subsequently, the question was repeatedly asked why IPTC/XMP could not have been made a superset of IPTC/IIM, and your question as to why pre-existing IPTC/IIM data could not be read into the XMP schema as a matter of course is an eminently sensible one.

Part of the problem is that the IPTC allowed Adobe much too free a rein, so that, for example, a number of the IPTC fields/tags are actually recorded in the Photoshop schema, and not in the IPTC Core schema. This may not make much difference to you and me on a day-to-day basis, but it adds complexity and confusion, and gives our beloved camera makers yet another reason to treat IPTC data as if it were a dose of bubonic plague.

All in all, this is a pretty good case study in the rule of life that, if stupidity can be the cause, you should suspect that first. Everyone loses from the current situation, and users lose out most of all. Should the parties get together and sort out the mess? You bet they should. Will they? As Clive says, Hmmm.

David

[dhackney]

All,

I organized the meta data audit into a PDF document here:
http://www.hackneys.com/travel/docs/metadataaudit.pdf

Doug

[dhackney]

All,

I expanded the meta data discussion into an overview document.

Doug


The purpose of this document is to educate and inform digital photographers about the basics of digital photography meta data. It seeks to explain the primary underlying causes of common meta data challenges, issues and problems.

I welcome corrections, feedback, etc.


Digital Photography Meta Data Overview


Table of Contents

1 Introduction...................................... 2
2 Meta Data Defined..............................2
3 Meta Data Location.............................5
4 Viewing Meta Data..............................6
5 Meta Data Types and Purposes.............7
6 The Meta Data Challenge.....................8
7 The Five Causes of Confusion...............8
8 The Customer's Perspective.................15
9 The Software Developer’s Perspective...15
10 Recommendations..............................16
11 Summary.........................................16
12 Resources........................................1 7



1 Introduction

Winston Churchill’s immortal description of 1939 Russia as “A riddle wrapped in a mystery inside an enigma” is equally true of the current state of digital photography meta data.

Most casual digital photographers never come face to face with meta data. They take their photos, they upload them to a web photo sharing or social networking web site, they may even print a photo occasionally, and at no time are they even aware that digital photography meta data exists. In many respects, they are the lucky ones.

Serious amateur and professional photographers wrestle with meta data as part and parcel of their daily workflow. Meta data problems, flaws, corruptions and disasters are a regular part of photography forums and discussion groups. Challenges can be as simple as one tool displaying a photo’s caption/description while another will not. Or, they can be as catastrophic as the loss of thousands of photos’ meta data, often painstakingly entered over weeks, months or years.

Meta data, defined as “data about data,” can be a very technical, daunting subject. In practice and application in digital photography, it has often been an unmitigated disaster.

Even at its young age, digital photography has experienced multiple meta data standards. Each of these standards has been ignored, corrupted or “enhanced” by digital photography software vendors. There has been no backwards compatibility or standardized mapping established to bridge from one standard to the next. The lack of standards regarding the labeling of meta data information has led to a plethora of conflicting terms used to label the very same piece of meta data, spawning mass frustration and confusion among users.

Unfortunately, there is no immediate relief in sight for serious amateur and professional photographers. If anything, the meta data anarchy and chaos created by the software vendors promises to increase.

The only hope for those tasked with unwrapping the meta data riddle lies in better understanding digital photography meta data. Only by understanding meta data can we accurately diagnose the challenges and build workarounds.


*****

Full document is here: http://www.hackneys.com/travel/docs/metadataoverview.pdf

[hockeyshooter]

I've downloaded ExifTool and tried your batch process, Doug, specifically just to copy the IPTC artist and copyright fields into the corresponding EXIF fields:

# write the image creator
-XMP-dc:Creator > EXIF:Artist
#
# write the copyright statement
-XMP-dc:Rights > EXIF:Copyright

Unfortunately, the process also changes the Windows datestamp of the file - which I don't really want to happen.

Chris - could you not modify either Downloader Pro or BreezeBrowser Pro so that by default, at the very least the IPTC Copyright Notice field is copied into the EXIF Copyright field? There are a number of PHP scripts and PHP-based image libraries out there that can read EXIF data, but not IPTC data, and I think its important that the EXIF copyright field should be correctly populated.

Chris.

[dhackney]

Chris,

I'm not an expert on exiftool, so I can't say offhand if it has this capability, but it would be worth checking.

Try the following:

Download the full pearl library version and unpack it - Image-ExifTool-7.25.tar
Navigate to the HTML folder.
Open the file: exiftool_pod.html. Sample location: C:\Downloads\exif tool\Image-ExifTool-7.25.tar\Image-ExifTool-7.25\html\exiftool_pod.html
Review the documentation for a flag that will not alter the windows date/time stamp.


I'm in a location with little to no bandwidth, so I don't have the capability to fully research a solution for you.

Hope this helps.

Your suggestion for Chris is super. Hope we all can enjoy that capability at some point.

Doug

[hockeyshooter]

Thanks - from the help file, the option is:

-P
Preserve date/time of original file (FileModifyDate) when writing.

That's an upper-case P - and I can confirm it does work. The line in the batch file thus becomes:

c:/exiftool -tagsFromFile "@" -@ bbp-xmp2exif.args *.jpg -overwrite_original -k -P

I use Exifer to confirm.

[hockeyshooter]

I wondered if it would be possible to run Doug's batch file on a folder-full of images without having to either copy the batch and arguments files into the folder, or even to use a DOS command-line window.

I put both the batch file and the arguments file into the root of C: and modified the batch file to contain just the following:

c:/exiftool -tagsFromFile "@" -@ c:/bbp-xmp2exif.args *.jpg -overwrite_original -P -v1

If you then open an Explorer window containing the images you want to tag, select them all, right-click, choose Open With and then using Choose Program, pick the batch file. This will process all the files for you. It appears to work on folder on other drives too.

Note that when you have followed this procedure once, Windows remembers the path to the batch file in the Open With submenu - so its far quicker the second time around.

Note that I am only following this procedure for images I intend to upload to the web - to Flickr (http://www.flickr.com/photos/hockeyshooter/), Picasa (http://picasaweb.google.co.uk/ChrisValentineOU), or whatever - since web-based systems tend rely on and (sometimes) preserve EXIF data.

Chris.

[Chris Breeze]

The way EXIF is structured makes it very difficult to add new fields or to expand existing ones without damaging the existing data such as MakerNote fields. Some apps avoid this problem by discarding the MakerNote and some do their best but end up corrupting it which can cause nasty problems for apps that read MakerNotes. XMP data has the advantage that it can be parsed and rewritten with extra data without having to know what every tag means. Unfortunately there are still plenty of apps that still can't read XMP data...

[hockeyshooter]

When I was playing with the exiftool program, I got the impression that it binned the maker-specific fields for any operation, thus "leaving space" for modifications or additions of any of the "standard" tags. But I'm not sure that's the case - perhaps it depends on which fields you change and even on how much text you put into them.

What a minefield EXIF has become!

Chris.