Salta al contenuto principale


Unfolding now: news.ycombinator.com/item?id=3…

- openwall.com/lists/oss-securit…
- github.com/tukaani-project/xz/…

An incredibly technically complex #backdoor in xz (potentially also in libarchive and elsewhere) was just discovered. This backdoor has been quietly implemented over years, with the assistance of a wide array of subtly interconnected accounts:

- github.com/tukaani-project/xz/…
- bugs.debian.org/cgi-bin/bugrep…
- github.com/jamespfennell/xz/pu…

The timeline on this is going to take so long to unravel

#security #linux

Questa voce è stata modificata (8 mesi fa)

reshared this

in reply to Evan B🥥ehs

boehs.org/node/everything-i-kn…

I have begun a post explaining this situation in a more detailed writeup. This is updating in realtime, and there is a lot still missing.

#security #xz #linux

Questa voce è stata modificata (8 mesi fa)
in reply to Evan B🥥ehs

I really hope that this causes an industry-wide reckoning with the common practice of letting your entire goddamn product rest on the shoulders of one overworked person having a slow mental health crisis without financially or operationally supporting them whatsoever. I want everyone who has an open source dependency to read this message mail-archive.com/xz-devel@tuka…

reshared this

in reply to Glyph

"I never thought a sophisticated APT would backdoor *my* volunteer-maintained infrastructure that I got for free" sobs entire industry who voted for the "volunteer-maintained infrastructure that I get for free with no defense against sophisticated APTs" party

reshared this

in reply to Glyph

@glyph please note that we are ALSO no fans of the "subsume free software into capitalism" solution that corporate and statist rhetoric has been pushing for a couple years now
in reply to Irenes (many)

@irenes @glyph It's tricky to avoid the challenge that arises from the problem that (1) producing free software is work and (2) the workers live in a capitalist society and (3) the workers therefore need to pay for food and shelter.

Verily, there is no ethical consumption under capitalism.

in reply to Peter Brett

alas, there is no ethical compression under capitalism

reshared this

in reply to Peter Brett

@krans @glyph sure. well, so the reason we personally call the thing we do "free software" is precisely to highlight the point that our own goal in publishing stuff without charge is very much to work towards a world without that problem, by creating something that exists as far outside it as we can manage (not all the way - obviously we have the free time to do that because of our other privileges)
in reply to Irenes (many)

@krans @glyph people publish their work without financial cost for a long list of reasons, we don't speak for anyone but ourselves (Irenes) here
in reply to Irenes (many)

@krans@me.uk @glyph It's free like freedom, not like beer, and I can relate. I'm not dependent on donations for my livelyhood, but I put like ~5-10 hours into this today, it costed me $5.30 in hosting fees, and not one of the 35000 viewers donated. That's ok, I didn't have donations till recently, but this is what happens in the FOSS community. People like to be paid for their work, that just doesn't always happen. The fact people put free stuff out doesn't mean they don't want funding.
Questa voce è stata modificata (8 mesi fa)
in reply to Evan B🥥ehs

not free as in free speech but free as in free association of equal producers.

If some extract all the value produced that's not free association of equal producers.

Questa voce è stata modificata (8 mesi fa)
in reply to Irenes (many)

@irenes @glyph I thought it was called "free software" because users are allowed to do whatever they want to with it including modifications, not because it's provided free of charge.

The founders of the Free Software movement were Libertarians, not Socialists (unfortunately).

I guess we were talking at cross purposes — sorry.

in reply to Peter Brett

@krans @glyph we're very proactive-death-of-the-author about this. the FSF has failed to provide ideological leadership due to RMS's top-down style, but many of the ideals are good ones and it's the job of the current generation to renew the movement if we want our children to be able to enjoy its fruits the way we did
in reply to Irenes (many)

@krans @glyph but you're right, of course, it's a valid point. we just don't think trying to coin a new term would be useful, if anything it would be a distraction from the cultural work that matters
in reply to Irenes (many)

@krans @glyph we see it as important that our work be free-as-in-speech, yes, but it is also very much free as in we absolutely refuse to ever ask for or accept money for it (outside the scope of our day job)

again, yes, serious privilege on our part

in reply to Irenes (many)

@irenes @krans @glyph > as in we absolutely refuse to ever ask for or accept money for it (outside the scope of our day job)

Speak for yourself lol

in reply to Evan B🥥ehs

@krans @glyph yes, we are speaking for ourselves (just Irenes, not you). as we already clarified up-thread, in our display name, and in our bio, we are plural.
in reply to Glyph

@glyph currently adding iframe infrastructure to embed this toot into my site
in reply to Evan B🥥ehs

this looks phenomenal, is it open source? I miss embedding tweets and this looks even better than that did
in reply to Glyph

@glyph Yeah, Mastodon has it's own embeds but I wanted it to fit in with the site. It's DIY and very hastily banged together: github.com/boehs/site/commit/a…
in reply to Evan B🥥ehs

@glyph Very cool! Any chance you might be willing to share that particular snippet - even just the HTML structure - under a less restrictive license than AGPL? It seems like the kind of thing I would love to use or adapt on my own site, but I don't want to (and probably legally can't) share my site and all the services it uses under that license.
in reply to David Zaslavsky

@diazona @glyph sure. I've licensed this file under MIT

github.com/boehs/site/commit/6…

if you use it commercially consider liberapay.com/e/ but obviously there is no legal obligation here

in reply to Glyph

@glyph I'm frustrated that big tech's efforts to increase core library security are "your project is too popular, you must use 2FA" and "the best reverse engineers in the world will find your bugs and put you on a 90 day disclosure deadline" and not "here is $100K/year and benefits to keep doing what you're doing at your own pace."

reshared this

in reply to Geoffrey Thomas

@geofft
> I'm frustrated that big tech's efforts to increase core library security are […] not "here is $100K/year and benefits to keep doing what you're doing at your own pace."

Have you *met* big tech? They're very far down my list of people that I want empowered to make such decisions.

Yes, there are workers I want to get that salary to keep doing demonstrated good work at their own pace.

But you propose Microsoft or Google or Amazon should choose who gets that? Please no!

@glyph @eb

in reply to bignose

@bignose @geofft @glyph I think the argument is that if big tech is profiting immensely off the backs of free software developers, those developers should be compensated fairly
in reply to Evan B🥥ehs

> those developers should be compensated fairly

I've already agreed to that, yes.

> if big tech is profiting immensely off the backs of free software developers

There you get closer to where I think the problem is.

I definitely don't want any private for-profit big tech firms deciding who gets compensated (and, necessarily, who does not get fair compensation) for work that benefits us all.

@geofft @glyph

Questa voce è stata modificata (8 mesi fa)
in reply to Geoffrey Thomas

@geofft @glyph I'm certainly not disputing that it's a real problem that that doesn't happen more often, but isn't there some precedent for big tech companies hiring people to work on specific open source projects? So it's not totally unheard of
in reply to David Zaslavsky

@diazona @geofft @glyph there’s a lot of precedent for hiring maintainers of top-level programs whose brand (for lack of a better term) has reached the level of awareness of a C-level with a hiring budget. Collectively pooling money to help the projects C-levels have never heard of… has a much weaker track record. We’ve been trying to tackle it at Tidelift for a while and suffice to say I’ve definitely had a lot of “but it can’t happen to me” conversations.
in reply to Luis Villa

@luis_in_brief @diazona @glyph Yeah that resonates with my experience. People like GvR get hired (which is great!) but there's a whole dependency stack underneath. Their maintainers often have a strong résumé to get hired for a normal big tech job at a company that uses the language/ecosystem/etc. but not necessarily for maintaining the project as their job. Sometimes the job is even "build something similar for an internal non-OSS ecosystem."
in reply to Geoffrey Thomas

@geofft @luis_in_brief @diazona there are layers and layers to this. Famous maintainers get hired more than critical maintainers. And maintenance is important but how do you pay for the commons of *new* projects? The tidelift model gets us part of the way there, because these costs need to be aggregated and there needs to be some kind of oversight, but even if they were universally adopted (and that is far from true) there are so many missing pieces
in reply to Glyph

@glyph @geofft @diazona “Famous maintainers get hired more than critical maintainers.” Owwwwwwww.
in reply to Luis Villa

@luis_in_brief @glyph

I am way overdue in finishing and publishing my negative review of Eghbal's "Working in Public" but one of my critiques is that she basically concludes that maintainers need to become famous and use Substack/Patreon to crowdfund (from individual donors) in order to sustain their work. Which really doesn't fit what we have found in critical FLOSS infrastructure IMO.

@geofft @diazona @eb

in reply to benwis 🦀

@benwis @brainwane @luis_in_brief @glyph @geofft @diazona @djc there’s some website (I forget what it is) that basically you pay x amount of dollars and it audits your entire dependency tree and attempts to pay maintainers proportionally. Unfortunately iirc it was kinda flawed but I think it’s a solid idea
in reply to Evan B🥥ehs

@benwis @brainwane @glyph @geofft @diazona @djc you’re thinking of Back Your Stack, probably.

On a more sustainable (read: commercial) basis, I co-founded tidelift.com to do exactly this.

in reply to Luis Villa

@luis_in_brief @benwis @brainwane @glyph @geofft @diazona @djc oh that’s sick, it’s so funny that you never know who you’re speaking to on here lol. Congrats on how successful tidelift has been :)
in reply to Evan B🥥ehs

@benwis @brainwane @glyph @geofft @diazona @djc Thanks! admittedly on a day like today, mostly I'm focused on how many projects we can't yet cover.

So, yeah, send people our way!

in reply to Glyph

@glyph god, imagine how betrayed Lasse Collin (maintainer) must be feeling rn
in reply to paul

@trespaul @glyph it's 2am in finland right now. I do hope they made it to sleep before they saw this
in reply to paul

@paul Yeah, wow.

"Recently I've worked off-list a bit with Jia Tan on XZ Utils and
perhaps he will have a bigger role in the future, we'll see."

That's two years ago and I'm assuming it's the Jia Tan that contributed the exploit this month.

mail-archive.com/xz-devel@tuka…
/via mastodon.social/@glyph/1121809… @Glyph

@Evan Boehs


I really hope that this causes an industry-wide reckoning with the common practice of letting your entire goddamn product rest on the shoulders of one overworked person having a slow mental health crisis without financially or operationally supporting them whatsoever. I want everyone who has an open source dependency to read this message mail-archive.com/xz-devel@tuka…

in reply to Evan B🥥ehs

so is there any evidence that "Jigar Kumar" is a real person? He certainly doesn't hold back from laying on the guilt tripping
mail-archive.com/xz-devel@tuka…
in reply to fraggle

@fraggle He's not. Like beyond reasonable doubt. Just guilt tripped and ran off into the sunset
in reply to Evan B🥥ehs

I kinda doubt Dennis Ens is either. He initially opened the thread, likely to plant the idea that xz needs a new maintainer. He then came back to guilt trip "I am sorry about your mental health issues, but...the community desires more"

EDIT: I initially misread "desires" as "deserves". I think my point still stands though

Questa voce è stata modificata (8 mesi fa)
in reply to Evan B🥥ehs

is anyone in touch with the original maintainer?

Given the pressure they were subjected to I don't want to imagine what the story that unfolded in the past view hours does to them.

in reply to Evan B🥥ehs

great work on this timeline. Here's Jai Tan's account trying to get xz 5.6.1 into ubuntu lts 24.04 yesterday. Beta freeze is on Monday. bugs.launchpad.net/bugs/205941…
in reply to Evan B🥥ehs

(Can you - if it's not too hard - also show the "last updated" timestamp in UTC?)
in reply to njsg

@njsg I'm updating this manually so unfortunately hmm I can try
@njsg
in reply to Evan B🥥ehs

comment on the libarchive PR is incorrect. The change does do what it says, in addition to changing safe_fprintf to fprintf. It added a strerror(errno) to the output, to address github.com/libarchive/libarchi…
in reply to Evan B🥥ehs

It's interesting that JiaT75 commits are logged with +0800 timezone, but their actual Github interactions appear to have been almost completely between 12.00 UTC and 18.00 UTC:

play.clickhouse.com/play?user=…

in reply to berdario

@berdario I've received tips about an associated linkedin account where the perpetrator claims to live in california. I've debated about if I feel comfortable publishing this, but I think i will.
in reply to Evan B🥥ehs

Looks like GitHub took down the repo, so some links don’t go anywhere anymore. I’d love to see the diffs if there’s an archive/screenshot ☺️.

Thanks for the write up! This is wild…

in reply to Evan B🥥ehs

I should note that Lasse also has some suspicious activity recently, such as github.com/plougher/squashfs-t… --and also may be currently affiliated with Jia in some capacity, as per tukaani.org/about.html
in reply to endrift

@endrift This is interesting. I've also heard the absolutely crazy theory this is all Lasse's doing given the timezones roughly match up and I mean I guess it's possible you could orchistrate all this conversation yourself. Would be criminal mastermind level stuff if so, but I want to give them the benefit of the doubt.
in reply to Evan B🥥ehs

@endrift the only thing Lasse would gain out of that is that they'll likely continue to be maintainer now that this has been discovered. I don't think that's close to enough reason for them to orchistrate all the conversations, so I'm not buying that theory tbh

It seems more likely that Lasse got bought out (or convinced) by Jia, after Jia gained his trust

That said, the most likely explanation imo is that Lasse was telling the truth in the PR description and wanted RISC-V filter 1/2

in reply to Evan B🥥ehs

thanks for writing this. they tried to change the URL to the xz subdomain on LKML too, see lore.kernel.org/lkml/202403201…

there's an 11-part patch posted at the same time but this specific patch wasn't part of it for some reason.

in reply to Evan B🥥ehs

“As I have hinted in earlier emails, Jia Tan may have a bigger role in the project in the future. He has been helping a lot off-list and is practically a co-maintainer already. :-)”

mail-archive.com/xz-devel@tuka…

This is really sad. I’m feeling bad for Lasse Collin.

in reply to Evan B🥥ehs

a friend just shared this with me. (instant follow) Thanks for writing this up. 🕵️
in reply to Evan B🥥ehs

aner news, another subtle thing fixed: chaos.social/@danderson@hachyd…


The poor original maintainer of xz is on it now, and has already found another "fun" thing: git.tukaani.org/?p=xz.git;a=co… . The configure check for enabling the Landlock sandboxing facility was subtly broken, so that Landlock support would never get enabled. The original malicious commit landed around the same timeframe as the main backdoor, also at an abnormal time of day compared to the new maintainer's historical activity pattern.

in reply to rugk

@rugk that’s already noted, thanks for letting me know though :)
@rugk
in reply to Evan B🥥ehs

the kernel news are not, though, or do I miss sth.? chaos.social/@olov@mastodon.wo… chaos.social/@rugk/11218182922…


thanks for writing this. they tried to change the URL to the xz subdomain on LKML too, see lore.kernel.org/lkml/202403201…

there's an 11-part patch posted at the same time but this specific patch wasn't part of it for some reason.


in reply to rugk

@rugk yeah I’ve seen that floating around for a while and I just haven’t had an opportunity to fully understand the implications of it
@rugk
in reply to Evan B🥥ehs

it would be useful to replace the commit links with non-Github ones given the Github repo was shut down. The URLs are like this: git.tukaani.org/?p=xz.git;a=co…
in reply to Evan B🥥ehs

Why am I the only one wanting to give the actor some roses for sticking to a project for 2 years?
Questa voce è stata modificata (8 mesi fa)
in reply to Evan B🥥ehs

what's really surprising is the same thing not happening more often. Or we are all just very unaware since the pay off to do these kinds of things is so huge specially for state sponsored attacks or others with enough money and effort to put into it.
Unknown parent

mastodon - Collegamento all'originale
Evan B🥥ehs

@james @trespaul @glyph

I understand why somebody is trying to get in touch with them, and also I'm not sure it's appropriate. The GitHub account is already suspended. The damage has already been done. Now they are being pulled back into something they clearly don't want to engage with

in reply to Evan B🥥ehs

@james @trespaul “somebody” in this case appears to be CISA, and like… that’s a federal agency, there is likely to be a law enforcement consequence or potentially national security response here. I feel very sympathetic to a burnt-out maintainer who had this dropped in their lap but there is an overriding public interest at this point.

People who are just curious should for sure steer clear though, this is likely to be a complete nightmare for everyone involved without extra emails

in reply to Glyph

@glyph @james @trespaul No I get that, it just sucks. Also mind that "national security" should mean nothing because the author is finnish. I hope the US government kindly asks "what do you know" and then leaves them alone, but we all know that ain't happening
in reply to Evan B🥥ehs

I've read about this a few hours ago. The seriousness was downplayed, luckily you've pointed us to a good source of info. Thank you
in reply to Evan B🥥ehs

love how it was buried in m4, a macro language that looks like jibberish on a good day — only well understood by three people that work on Autoconf (everyone else just copypastas), maybe someone in a basement hyper-tuning their sendmail config, and apparently whoever this guy is.