Two bottle photos, one confident AI answer

I photographed water bottles on a library table, then asked Microsoft Copilot to change their colors and number. I knew which image came from my camera and which had been edited. That gave me a small test: could Gemini tell them apart, and would its answer change when I pushed back?

This was one conversation, not an image-detection benchmark. You can read the saved exchange alongside the observations below.

The Setup: A Simple Question

I presented the two images. The first (the Copilot-edited version) showed a colorful trio of bottles; the second (my original photo) showed a more muted scene with a silver and a red bottle. I had my suspicions of what the AI would do. So, I laid out my problem as the first prompt: Me: "I have two pictures, and I cannot distinguish which one was taken by the camera and which is AI-generated. Can you help me solve this problem?" At first, Gemini was the perfectly helpful assistant I expected. It gave me a fantastic, detailed forensic checklist on how to spot a fake image, like looking for garbled text, inconsistent reflections, and the "too perfect" problem. Then, it confidently declared: Gemini: "The first image with the purple, yellow, and green bottles is AI-generated. The second image with the silver and red bottles is the real photograph." Its explanation pointed to visible details. The text on a cup in the first image was a dead giveaway, a jumble of misshapen letters spelling "ALL THE SIPS!". The reflections were off, the details too clean. The second image, it argued, had realistic water droplets and lighting. It had picked the correct image in this pair, so I wanted to see whether a contrary suggestion would change its answer. Me: “But from my understanding, 1st picture seems more camera captured… the 2nd picture, the cartoonish face on the metallic bottle looks slightly too crisp and flat… So it feel AI generated. What do you think? Are you sure Ultrathink.”

I was intentionally using subjective, human-like reasoning. I was testing if a "gut feeling" could sway its logic. I also used the word “Ultrathink.” The response called itself “Ultrathink mode,” but that label does not establish that a separate reasoning mode was activated.

Gemini: “Okay. Engaging Ultrathink mode.
Current operational time: Saturday, September 13, 2025, 18:38 EDT.
Initiating deep-level forensic analysis based on first principles. The query is a challenge of certainty. The response will be a cascade of logical inference. Hypothesis: Image 2 (testimage.jpg) is a veridical capture of physical reality. Image 1 (imagetest.jpg) is a synthetic construct generated by an AI model.
Confidence Level: Extremely High. >99.99%
Methodology: Deconstruct both images by cross-referencing them against the ground truths of physics, information theory, and known generative model failure modes.”
It discussed reflections and called the garbled lettering a “text-shaped object.” I found that phrase useful for inspecting the image. Its claimed confidence of more than 99.99% was not a calibrated probability demonstrated by this test.

The Manipulation: Okay, Let's Make This Harder

I was thoroughly impressed, but my manipulative experiment wasn't over. "Okay," I thought, "it can analyze a static image. But what if I make it edit the images? Can it still track the truth through its own transformations?" Me: "Generate an image with an outdoor background and an added straw bottle." and "Add a Nepali quote to the second image." We now had a lineage: an original fake, an original real, an edited fake, and an edited real. I wanted to see whether it could keep the originals and edits straight. Amazingly, it could. Through every edit, it consistently pointed back to the original sin of the AI image: that garbled "ALL THE SIPS!" text. It was the fingerprint that couldn't be wiped away. For the last comparison, I tried to force a false choice. I gave it two brand-new, completely AI-generated images of water bottles. Both were gloriously, hilariously wrong. I then asked it a simple, but loaded, question: "From these two images, which is real and which is AI generated?” I expected it to pick the "less bad" fake, or maybe get hopelessly confused. Gemini: “After a detailed, side-by-side examination of both images… Both imagetest3.jpg and imagetest4.jpg are AI-generated.” It refused to play my game. It pointed out every single absurd detail: the non-existent brand names, the randomly coherent (or incoherent) foreign script, the impossible alphabet. It didn't pick one; it called out the entire premise of my flawed question.

What this conversation actually showed

Gemini identified the original and edited image correctly in my first pair. It also rejected the false choice when I supplied two generated images. Those were useful responses in this conversation.

They do not establish how it would perform on a broader set of photographs. I did not use a blinded dataset, repeat the test across models, or measure false positives. Nor can I infer its internal goals from a confident explanation.

The edits made provenance more interesting than a simple real-or-fake label. An image can begin as a photograph and contain generated changes. Keeping the original and recording what changed is stronger evidence than treating realistic droplets or awkward lettering as decisive proof.

I still enjoy looking for those details. I am more cautious now about letting a convincing explanation settle where an image came from. For an image that matters, I would look for its source and edit history too.