Category Archives: Tests

Equipment, Ideas, Tests

Experiment #2 Revealed

2 Comments

Yesterday I posted “Experiment #2: What do you see?,” in which I shared six image files comprised of three identical pairs of images and asked volunteers a) whether they saw any differences among them when viewed in their web browsers, b) to describe any differences that they noticed, and c) to try to identify the pairs of identical images. As describe in the original post, all of the images came from the very same source file – e.g from a single exposure – and were processed identically with the exception of one variable that was not identified.

Here are the 100% magnification crops from the three source images:

The differences among them are obviously in the amount of noise that was added to the image. No noise was added to the first image – any noise there was in the original capture. 10% level “Uniform” noise was added to the second image in Photoshop. 20% “Uniform” noise was added to the third image in the same way.

While I could have varied camera ISO to produce actual camera-generated noise, doing so would have also produced other variations in the images that would have given secondary and possibly misleading cues as to the differences between images. This most certainly would have affected part c) of “the question” as outlined above. While recognizing that noise added in post is not going to be exactly the same as noise produced in camera, I did try to ensure that the noise would at least be of a type and level that would clearly cause concern if the camera did produce it.

The soft photograph was chosen to avoid masking the noise with a lot of other sharp detail – this image provides very smooth gradients from black to white, where noise is typically easier to detect. I also chose this image because it is nearly – but not quite – monochromatic. This meant that I could increase the effect of the noise by using color noise rather than limiting to monochromatic noise – and that the color noise would tend to be more visible against the nearly monochromatic background.

While quite a few folks reported that they didn’t see any difference among the image when viewed in their web browsers – and, frankly, this did not surprise me – some did report noting differences. Test subjects have been known to both correctly identify real differences… and to think they have seen real differences where none existed. With that possibility in mind, I was interested to see how accurate the “perceptions of difference” might be, hence the challenge to find the pairs of identical images. The idea here is that if one can really see differences between images that one should then be able to categorize the images accurately based on those differences. I won’t comment here on whether any individuals were right or wrong, but here are the six images grouped as identical pairs.

No noise added:


10% noise added:


20% noise added:


A good number of readers asked, “What is the point?” A few even were upset at a test of something they regard as settled – e.g. that noise and other small artifacts become imperceptible when a large image is reduced to typical web sizes. (In this case each pixel in the jpgs is the average of close to 100 pixels in the original file.) However, I can say for sure that this issue is not resolved in the minds of all photographers nor in the minds of many who are making purchase decisions about cameras for themselves or for others.

My thesis was essentially that very significant amounts of noise that would be clearly visible in large original files at 100% magnification will be indistinguishable from files that have far less noise but are otherwise identical when the files are reduced for typical web site use.

A direct “point” might simply be that if you reduce 21MP full frame photographs containing large amounts of noise to 600 pixel width high quality jpg files viewers of the images on the web seem unable to reliably notice the differences in noise levels. You could reasonably extrapolate from this that if your main reason for shooting photographs is to share them on the web, noise levels in the camera may not be an important decision point for you as you shop. Though you cannot extrapolate the following directly from this test, I believe that shooters who mainly share jpg images or perhaps make letter-size prints will not see any significant image quality benefits from getting really high-end cameras. If noise levels as different as those found in this experiment cannot be discerned then the quite small differences in noise between two brands or models of camera are likely to be completely insignificant in images viewed online at typical dimensions. (If you make very large prints on a regular basis then your issues will be different.)

For my part, even though I created the images, I cannot reliably tell them apart by looking at them! When I look up my record of which image was treated which way I think I can see the difference, but I’m pretty certain that if I had to try to pair the identical images I would be unsuccessful. (Note: if you view the images one above other on this page you will think they are different due to viewing angle differences on your monitor. Go to the original post to see them displayed successively in the same location on the screen.)

(Experiment #1 tried something similar, though in that case the variable was the “sharpness” of the original image file.)

Tests

Experiment #2: A Hint

2 Comments

Here are the three source images:

The differences among them are obviously in the amount of noise that was added to the image. No noise was added to the first image – any noise there was in the original capture. 10% level “Uniform” noise was added to the second image in Photoshop. 20% “Uniform” noise was added to the third image in the same way.

The soft photograph was chosen to avoid masking the noise with a lot of other sharp detail – this image provides very smooth gradients from black to white, where noise is typically easier to detect. I also chose this image because it is nearly – but not quite – monochromatic. This meant that I could increase the effect of the noise by using color noise rather than limiting to monochromatic noise – and that the color noise would tend to be more visible against the nearly monochromatic background.

The original set of 6 jpgs includes two from each of the three versions shown here. For last-minute fun you could still visit the comment page and try to a) determine whether or not the differences are visible and b) which pairs of images are the least noisy, more noisy, and most noisy.

More information to come later this evening.

G Dan Mitchell is a California photographer and visual opportunist. His book, “California’s Fall Color: A Photographer’s Guide to Autumn in the Sierra” (Heyday Books) is available directly from him.

G Dan Mitchell: Blog | Bluesky | Mastodon | Substack Notes | Flickr | Email


All media © Copyright G Dan Mitchell and others as indicated. Any use requires advance permission from G Dan Mitchell.

Tests

Experiment #2: What do you see?

25 Comments

In this experiment I invite you to view a sequence of six jpg files that include different versions of the same original photograph.

  1. All six images seen at the link began as the same original source file – a RAW file from a high quality full frame camera.
  2. Three versions of this file were created, with each altered in a particular way I won’t yet reveal in advance – let’s call them versions , , and .
  3. Each of the three versions was then duplicated – in other words two identical copies of version were created, two identical copies of version were created, and two identical copies of version were created.
  4. The six files were randomly assigned slightly different file names so that in the end we have File A, File B, File C, File D, File E, and File F.
  5. The six images were then sequenced into a “slide show” version that automatically switches between them, but which also allows the viewer to select them individually.

The Experiment

After you view the linked files in your web browser as presented at the linked page I would like you to consider three questions:

  1. By viewing the files in your web browser as presented here are you able to perceive any visual differences between the files aside from the letters A-F indicating the different images?
  2. If you do perceive differences among them, to what would you attribute the visual difference?
  3. By viewing them in your web browser as presented, can you identify the pairs of identical images?

Click on this link to view the sequence of six presentations of the photograph. The sequence should open in a new window/tab when you click the link. Return here to report your results by adding a comment to this post. (Note: The link was briefly broken – apologies to those who got the missing page error. The problem should now be resolved. Direct URL: https://gdanmitchell.com/images/Experiment2/index.html )

Caveats and Warnings

  • I do not make any claims that this is a scientifically constructed experiment – let’s call it an informal investigation of a potentially interesting question.
  • If you view the comments of other participants before offering your own response you may be influenced by what they write. Therefore in the spirit of the experiment you should decide on your results before posting your response and before viewing the comment page – and resist the temptation to change your results after reading what others say.
  • The whole point of this exercise is to consider what you see when you look at these image in your web browser. If you want to participate, please limit your consideration of the images to viewing them in your browser on the linked page.
  • Feel free to look at the images as many times as you want or to switch among them manually. (You might want to mention the extent of your careful viewing in your comment.)
  • I’ll stipulate that if you download the files and carefully investigate them that you will possibly find differences whether you can see them or not. But don’t – this isn’t the point here. If you can’t resist doing this please keep your discoveries to yourself until the end. (I’ll also point out that I could do things in the files that might mislead you… ;-)
  • You are welcome to hold positive or negative opinions about the aesthetic quality of the photograph, but please keep those out of your response to this experiment.
  • I am not saying in advance what the difference are nor am I making any assumptions ahead of time about what the results will be. It is best if you do the same and keep an open mind.
  • I plan to reveal everything about the experiment in a day or two.

(This is the second in a series of visual experiments that I invite you to try. You can see the previous Experiment #1 and the followup post about the results.)

G Dan Mitchell is a California photographer and visual opportunist. His book, “California’s Fall Color: A Photographer’s Guide to Autumn in the Sierra” (Heyday Books) is available directly from him.

G Dan Mitchell: Blog | Bluesky | Mastodon | Substack Notes | Flickr | Email


All media © Copyright G Dan Mitchell and others as indicated. Any use requires advance permission from G Dan Mitchell.

Equipment, Ideas, Technique, Tests

Experiment #1: What do you see?

37 Comments

(Update July 30, 2011: Since the original test was published quite a while ago, I have added this link to the outcome of this test. Of course, if you want to give the test a try yourself, you should read the whole article, try answering the questions, and only after that come back here to follow the link to find out how your response compares.)

Yesterday I posted a series of three images in a couple forums I frequent along with a request that a few people try a little experiment with them. Here it is for anyone else who would like to participate.

Below on this page are three images presented at a typical web viewing size. The question has to do with what you see when you look at them in your browser. (Please read some additional material further down that explains why you must only look at the photos in your browser for the purposes of this exercise.) Considering all aspects of the presentation of the images, in the end there are three possibilities:

  1. all three images seem identical – e.g. there are no perceptible differences among them.
  2. all three images seem different from one another – e.g. each is visibly unique.
  3. two images seem identical but one seems unique – e.g. one is visibly different from the other two, and those two seem visibly identical

Before you give it a try, there are a few “conditions and warnings.”

  • You could easily cheat by opening the files and looking at EXIF or other data. But don’t. Or if you cannot resist, please keep your observations to yourself. I’ll stipulate that you could find differences in file parameters by looking there – but that isn’t the point in this case.
  • You should not assume that I am incapable of modifying EXIF and file size and so forth in ways designed to trick those who “cheat” and inspect the files directly. Nor should you assume that I have. Or that I have not. Or whatever… :-)
  • The question is not “are they three separate exposures or one exposure,” so if, for example, you think that the water looks exactly the same in all three images that, in and of itself, isn’t relevant to the question – and your assumptions may or may not necessarily be correct.
  • The question is not whether this is a good, bad, or indifferent photograph – I make no claims beyond the fact that it is used here as a test image.
  • The question is not “what would they look like at 100% magnification?” Interesting question, but here the question is just about what you see in the images as presented.
  • If you think that you see differences among them, the followup question concerns the nature of the visible differences. Note that the follow-up question is not “how might the differences have been produced?” Just describe what you think you see.
  • There is no “point” inherent in the exercise, though when I explain more after getting some responses you might or might not draw some of your own conclusions. For now, just compare what you see.

(Follow-up observation. Having access to server logs, it is interesting for me to note the percentage of people who share a response versus the number of page views… ;-)

(Note on 9/10/11: Especially on flat-panel monitors, the position of the image on the screen may create visible differences that are not in the source images. Try to view the samples in the same location on your screen to avoid being misled by this. Once you make your observations and read comments here and in the follow-up post, you may see that others noticed and commented on this.)

I have set this up so that clicking on each image will open it in a new window or tab – though it is a bit awkward in that you’ll need to manually return to this window after doing so. Once they are open in three tabs (best) or windows you can click between them to compare more carefully if you wish – but do stick to viewing them in your web browser since that is part of “the question.”

After viewing, leave a comment stating which option (1, 2, or 3) best describes what you see. In addition, if you select option 2 you might offer a brief explanation of differences you see. (Again, not analysis of downloaded files please! That isn’t the question and it spoils the “game” for other participants. ) If you select option 3 tell us which one is different from the other two and perhaps what you observe about the visual difference.

Here are the images (Click to open each in new windows if you would like – you’ll have to manually return to this window each time – or feel free to view them here “in line.”):

A.

B.

C.

Thanks for participating!

Dan