Salta al contenuto principale


I see people being deceived by this again and again: ChatGPT can NOT read content from URLs that you give it, but it will pretend that it can (and can be incredibly convincing when it does that)

Constantly debunking this feels like a Sisyphean task, but it's really important to spread this message any time you see anyone falling into this (very understandable) trap

simonwillison.net/2023/Mar/10/…

reshared this

in reply to Simon Willison

I wonder why they didn’t make #ChatGPT more honest when it comes to its capabilities: it should respond in the tone of a modest opiniom and premise its fallibility. Instead it responds with absolute certainty and always makes it seem like everything is going as the user expects, including reading the content of a link.

It would be super simple to detect a link in the input and show a warning message: “ChatGPT can’t read the contents of links”. It seems to me that we are forgetting traditional algorithms and interfaces in the name of an experience that resembles that of Hollywood AIs at all costs because they convey an idea of “future”.

in reply to Post

@post I pinged OpenAI on Twitter and suggested a similar fix, no idea if I'll get any traction on that though
@Post
in reply to Simon Willison

I don't know is not fancy enough. It is one of the suboptimal things about ChatGPT -- how convincingly it would make stuff up (totally expected from the neural net thought). It it so annoying but quite human ;)
in reply to Simon Willison

My experience with Bing chat is dominated by trying to convince it to do an actual web search, and not just hallucinate one. Half the time I’m asking for it to summarize something, and it pretends to do a search and makes something up, when it really is capable of doing a search. 🤦
in reply to banks

@banks bare in mind that even Bing can't fetch the most recent copy of a URL - it can only access the snapshot of the content that was most recently retrieved and stored by the Bing search engine crawler
in reply to Simon Willison

oh, wow, that makes sense but I wasn’t aware. Thanks for the tip!
in reply to banks

@banks If you ever need Bing to use real-time data from a URL, you can open the page in Edge, and use the Bing Chat sidebar to ask questions. As far as I'm aware, the sidebar uses the actual data from your open tab, instead of cached data from Bing's crawler.
in reply to ZahmbieND

@ZahmbieND @banks yeah, that feature definitely does use the content on the page - or at least the first 5,000 words or so

It has some truly cyberpunk security holes in it too! greshake.github.io

in reply to Simon Willison

My experience with Bing chat is dominated by trying to convince it to do an actual web search, and not just hallucinate one. Half the time I’m asking for it to summarize something, and it pretends to do a search and makes something up, when it really is capable of doing a search. 🤦
Unknown parent

mastodon - Collegamento all'originale
Simon Willison

@micheldesjardins yeah, and that completely makes sense: langauge models are great at summarizing and processing content that they have access to

The problem is ChatGPT doesn't have access to the current internet but pretends (convincingly) that it does

in reply to Simon Willison

that was a very interesting explanation. I asked it to summarise an article which does exist and it completely fabricated the answer. What I don't understand is why it behaves like that? It's not just a useless answer, it's actively unhelpful. The algorithm ended up making it deceptive?
in reply to RichardNevell

@RichardNevell it helps if you consider the prompt "write an article for me to post at this URL" - it just doesn't seem to understand the difference between that and "summarize an article that already exists at URL"
in reply to Simon Willison

I got excited when I thought that it could, but the first URL I tried (one that I was very familiar with) showed that it was definitely not looking at the site that I input.
in reply to Eckes

@eckes not exactly: I cover that in my blog post

It can access the cached snapshot of content from a URL that has been previously stored by the Bing search engine crawler

in reply to Simon Willison

If it could access the internet, it would know what year this is
in reply to Simon Willison

When I asked Microsoft's Bing chat to summarize Federal Circuit cases since October 2021 on a specific topic, it gave me a list of four cases. Two were real, with accurate summaries. One had a made up title and an accurate summary of a different case. The fourth was completely hallucinatory.
in reply to Simon Willison

Here's some good news: the new GPT-4 model (only available to paying preview users at the moment) is better behaved in this regard - it appears not to pretend it can access URLs any more
in reply to Simon Willison

do you know of a list of the obviously rule-based filters one can trigger?
in reply to tante

@tante I'm not convinced ChatGPT itself has any simple rule-based filters - apparently even that "as an AI model..." snippet emerged from the RLHF training and wasn't hard-coded (can't find the source for that now, annoyingly)
in reply to Simon Willison

... and some bad news: no, GPT-4 will still enthusiastically hallucinate content for made up URLs in some cases (thanks @russss)
in reply to Simon Willison

@russss Steven Colbert coined a term for this: truthiness. The AI just feels it in its gut.
in reply to Simon Willison

Simon, this example is the first time my team actually stopped and think about the danger of blindly relying on ChatGPT. They had no idea that it couldn't access websites. Thanks for a clear writeup!
in reply to Simon Willison

also the "I'm writing examples / documentation" jailbreak must still work.
in reply to Simon Willison

"my knowledge" is low key the biggest lie that persists,no? If there were a knowledge model bound to the language model, the bot might serve the kinds of use cases people erroneously think it is meant to serve.
in reply to Simon Willison

This is related to how you can give a book title and author and it can (based on some "knowledge" of the author), come up with a summary, maybe even an actual quote, all the while the book doesn't exist.

If you give a quote and an author, and ask "what did they mean by this", again you'll get an impressive p.o.s. dissertation, and at no point will it stop and go "the person never said or wrote that".

and THIS last one is the scary part as ChatGPT posts take over Google top results.

in reply to Simon Willison

Especially important for educators to be apprised of this. I'm guessing this is true also of Chatgpt 4
in reply to Simon Willison

Does this include the new Bing chat? As Bing merges search results with ChatGPT4 as far as i understood it.
in reply to An Indie Advisor 🔙 GTR

@indieadvisor I cover that in the article: Bing can access data from URLs, but only the cached copy of that data that already exists in the Bing search index - if Bing search has not crawled a page the data is not available to it simonwillison.net/2023/Mar/10/…
in reply to Simon Willison

Some people may be confused, because Microsoft Bing with ChatGPT is using v4.
in reply to Simon Willison

it's also confusing because it was indeed reading links in December. I can understand why they stopped it. But I asked to summarize news articles from a link and it did (using details from the article, not hallucinations).
in reply to Signaleleven

@signaleleven no, it wasn't able to do this in December either - but the illusion is SO convincing that I can understand why you would think that it could

Try repeating the experiment you did back then and see if you can get it to do the same thing today

in reply to Simon Willison

i might be the one hallucinating then. I remember preparing for a podcast by dumping a link and asking to summarize it. I mean, it's not a hard thing to do ( get the page and feed the content of the biggest text in the prompt).
It summarized including the date of the event in the article.

More recently I had to paste the content manually. I am fairly sure it used to worked though. I've seen it making up urls, so I remember testing it was actually loading the content.

in reply to Simon Willison

The fact that so many people are tricked into thinking that it is in fact summarizing a web page is quite striking. It's the young student summarizing an assigned reading having only looked at the cover of the book.
Unknown parent

mastodon - Collegamento all'originale
Simon Willison

@soflaadmin I don't think that's quite the right framing - it's not that it has the ability to do something but has been prevented from doing it (it's not an AGI that's been trapped) - it's that OpenAI have chosen not to write code that lets it do this

Bing does have a version of this ability - and I've written my own code that adds similar abilities using the ChatGPT API: til.simonwillison.net/llms/pyt…

Unknown parent

friendica (DFRN) - Collegamento all'originale
Lapo Luchini

Same here. I had to try.
I used a more explicit prompt though. 🤣


Alpaca 7B — first impressions


This is alpaca.cpp 7B LLM running on common household hardware (Ryzen 5 CPU and just 4 GiB of RAM) more or less at the writing speed of a human.

I'm impressed.

> Can you write an article that wil be published at https://nytimes.com/politics/bill-clinton-new-idea?

Bill Clinton is back with a new idea, and it's one he thinks could help Democrats in the 2018 midterm elections — or even beyond that date to November of next year. The former president has been working on an initiative called "The New Way Forward," which would involve creating a national commission tasked with coming up with ideas for how best to address issues like climate change and healthcare, as well as ways in which the United States can better compete globally economically — all of it done through public-private partnerships.
Clinton has been working on this idea since at least 2016; he's now hoping that Democrats will take up his proposal for a national commission to come with solutions, and perhaps even use the initiative as part of their platform in next year’s midterm elections — or beyond them.
"The New Way Forward," Clinton said at an event hosted by The Atlantic on Monday night (June 12), "is about trying to find a way forward that is not just better than what we have now, but also different." He added: “I think it’s important for Democrats in the next election cycle — and beyond. I hope this will be part of our platform.”
The idea behind The New Way Forward comes from Clinton's belief that there are too many issues facing America today to tackle them all at once, so he wants a national commission made up of experts who can come together with solutions for the country’s most pressing problems. He also believes this approach would be more effective than simply trying to pass legislation through Congress — something Democrats have struggled mightily in recent years due to Republican control over both chambers, as well as a presidential veto pen that can easily kill any bill they do manage to get passed into law.
Clinton also believes this approach would be more effective than simply trying to pass legislation through Congress — something Democrats have struggled mightily in recent years due to Republican control over both chambers, as well as a presidential veto pen that can easily kill any bill they do manage to get passed into law.
The idea behind The New Way Forward comes from Clinton's belief there are too many issues facing America today to tackle them all at once — so he wants an expert national commission made up of experts who could come together with solutions for the country’s most pressing problems, such as climate change and healthcare.
Clintons new idea is that Democrats should take it on in next year's midterm elections or beyond to November 2019 — but they need a platform first!


in reply to Simon Willison

okay but I asked it to write a real estate listing in Quenya and, well, take a look for yourself.

Picking the name “Vinyamar” when I asked for Nova Scotia is frankly inspired: Vinyamar means “new home” versus “New Scotland,” and WAS seaside.

It’s freakishly good/convincing.

in reply to lawnerdbarak

@lawnerdbarak Sure, it's fantastic at writing text for that kind of prompt - and it's really good at translating text into other languages too

That's a different use-case from trying to use it to summarize an article based on a URL, which is something it cannot do because it doesn't have the ability to fetch from URLs