Google has reportedly removed much of Twitter’s links from its search results after the social network’s owner Elon Musk announced reading tweets would be limited.
Search Engine Roundtable found that Google had removed 52% of Twitter links since the crackdown began last week. Twitter now blocks users who are not logged in and sets limits on reading tweets.
According to Barry Schwartz, Google reported 471 million Twitter URLs as of Friday. But by Monday morning, that number had plummeted to 227 million.
“For normal indexing of these Twitter URLs, it seems like these tweets are dropping out of the sky,” Schwartz wrote.
Platformer reported last month that Twitter refused to pay its bill for Google Cloud services.
Good. Hopefully they remove links to pinterest, quora and facebook too while they’reat it.
Once upon a time Quora had reasonably accessible information. Now I find it nearly unusable and only go there as a last desperate effort which is generally fruitless. Pinterest is annoying, but generally you can still view some content. What’s annoying is trying to download, copy, or isolate content there. If all you want to do is view an image, Google can typically still pull the image out and make it viewable from the search. The problem only arises if you try to go in and see the original.
Thank you for typing out my exact thoughts/experience. Saved me a lot of time, not having to put too many thoughts into words.
I’ll only add that, of course pintrest has workarounds… but I’ll usually spend more time looking for content that isn’t so “locked out”. Cause it’s a pain in my ass trying to go around it when I just wanna see examples of some random bullshit real quick. Sometimes I click a pintrest link without noticing and immediately go back to search, cause fuck that.
Quora I still click sometimes; cause it’s like a box of coconut-filled chocolates, save only one nutella-filled chocolate… you pretty much know what you’re gonna get
Yeah, Quora was somewhat decent, but now they force this arbitrary “All related” sort as the default setting for presenting answers and it shows answers for different questions which is nonsense. I’m looking for an answer specific to one question, so don’t show me stuff that appears somewhat related…
Please remove Quora links…
Every time I click a Quora link it takes me to a completely unrelated question to the one I clicked on.
I swear that site is doing this deliberately to make it seem like it has more traffic than it does.
You can omit search results from certain sites by adding -quora.com to your search if you’re feelings motivated
This functionality is currently broken on DDG and was broken on Google about one year ago. Search engines as a whole need complete overhauling
Is there a way to just add that flag to all of my searches? Quora is shit in general
I don’t know of a way to save it as a setting but there may be a plugin you could add to your browser
Pinterest in search results is totally cringy, but by itself it’s a very good place to find images you need and has very cool recommendation algorithms.
I have a chrome plugin to strip any Pinterest results from searches, it’s the absolute worst
What did pinterest do? Isn’t it just a bunch os pictures put together?
some cases with pinterest result is absolute garbage and might as well better to search on search engine image instead
Most underrated comment
Great. Do Pinterest next.
What’s the problem with Pinterest?
It fucks with Images results.
It really does, doesn’t it!?
Do people even use Pinterest nowadays?
I’m always staggered by how regularly it appears at the top of search results.
it’s still used as an image search engine / aggregator but probably not enough to pay the server bill
And Tiktok. And Quora.
It’s funny how Spez idolizes a guy who doesn’t pay his bills. It’s also funny watching the hate pplatform Twitter get absolutely destroyed by its owner. The internet in 2023 is a wild and crazily changing place.
At this point I’m half convinced spez was such an asshole to Reddits users because of how the community turned on his idol elon. I remember not longer than two years ago Reddit was basically musk circlejerk but this genius fucked up even that.
I think it’s even stupider than that. Spez didn’t realize he pissed off the Apollo dev, and the shock of that first Apollo post made him ashamed and angry. He’s kept a lot of vitriol aimed at the Apollo dev. Spez fucked reddit because he got mad.
That was just the tipping point. Reddit has been in a decline for a few years but we were all like frogs in boiling water but we accepted it because we didn’t know where else to go. RIF shutting down forced my hand and now I’m here. Hopefully to stay.
And for the better too. People will come to realize that there are better ways to socialize on the internet.
I can’t believe how fast the fediverse is growing thanks to both reddit and Twitter simultaneously imploding. I honestly welcome it, despite the growing pains this feels more like early 2000s reddit than the modern version ever did.
Can’t wait for genZ to welcome the return of spaces that aren’t optimized for monetization over communication first.
As someone who spent the last 13 years browsing reddit almost entirely with RIF, if it hadn’t been for third party services I’d have left when they added awards.
Now we need to do the same thing with YouTube and Twitch
That would be nice, but is at a whole other scale with bandwidth and storage space.
Although there is a fediverse alternative for Video Hosting: https://joinpeertube.org/
Are there any for video/voice chat similar to discord? I feel like there are split opinions on it as a platform as they’ve made so many changes with community servers, plus their early association to Charlottesville obviously leaves concerns…but there’s definitely tons of wholesome groups where I’ve made life-long friends. I love that it’s remained ad free but don’t see that lasting
There is https://matrix.org/ but the video/voice isn’t as nice as discord.
And this is interesting…
https://mastodon.social/@rodhilton/110657916419002616"rodhilton Rod Hilton @[email protected] I have some insider knowledge here that I wanted to share.
This is not happening because Google scrapes Twitter and is now unable to. Google has been a paying customer (with a special negotiated rate) of the Firehose API for nearly a decade. Presumably, that deal was still in effect, barring API rate changes having an impact.
So this decision is solely because the results can no longer be viewed by non-logged-in users.
Elon going to complain about another conspiracy going on while in reality it’s just that when crawlers are not able to open a certain URL they simply assume that the page doesn’t exist anymore. Google certainly didn’t “retaliate”, bots simply couldn’t find those pages anymore.
The headline is actually wrong. Google did not do anything to Twitter. Twitter fucked up their own SEO by removing access to its content.
This is correct.
Yeah, that’s a pretty easy and reasonable conclusion to come to if you think about if for more than five seconds. I’m not sure Elon has any toes left after he keeps shooting himself in the feet.
If your company cares about it’s SEO rankings, you don’t make changes like these without considering the SEO implications.
Not even a month later and said company rebranded itself without checking trademarks. Now we have “X”, a brand that non only risks infringement of quite a few registered eu-trademarks but didn’t even apply for an own eu-trademark…
Just like reddit. Looks like the Speztic and the Elongated Muskrat are caught in an ouroborus of like-minded stupidity.
Elongated Muskrat 🤣
deleted by creator
The latest in a seemingly never-ending series of self-owns. Apart from the stress it must put on their devs, it’s been entertaining
They have more than one dev left?
No it’s just one guy called Dev.
And he’s on a H1B visa and can’t leave.
They screwed up his name tag, because reasons.
His name is actually Dave. But there aren’t any other Dave’s working there anymore so he is ok with being called Dev.
Deb on the other hand is pretty pissed off.
Dude was just a marketing intern and is mad stressed and knows he’s in WAY over his head now.
Fake it 'till you make it.
Or the multibillion dollar company collapses into a heaping, flaming pile of slag as a result of your actions. One or the other.
My understanding is there’s group of people whose visa is sponsored by Twitter. If they leave the company, they may well have to leave the country.
That’s a big yikes and I hope they will find another way…
Coercing workers built America! They can’t leave… because of the implication.
Are you gonna hurt these workers?
Twitter still has devs?
Crawl issues I am sure but also user experience issues. Google is sensitive to sending visitors to sites where metrics indicate users do not, like bounce rates etc. I don’t use twt but if it is the case you have the be logged in to see anything now, a non-logged in user will click a link from Google hit a login page, and use the back button. I would assume Google will see that as a bad search result and use it less.
The word bots triggers the muskrat.
I wonder if Google prioritized recrawling all those Twitter links.
If I were making a web crawler, I would make it so that if a crawler finds a domain that appears to have changed dramatically or gone offline it will re-crawl the domain and flag already-crawled pages as potentially obsolete.
Elon, please buy Reddit and repeat your amazing ideas over there. You are so smart.
Spez is already doing his best work to fuck over the platform. Are you certain Elon could do any better?
Spez already said he admires how Elon is running Twitter.
I mean there’s admiring a ruthless mob boss - and then there is admiring a petty thief that keeps getting arrested and all his plans blow up in this face.
Spez is losing his tiny mind.
The best part is that Elon is proving to be a pro at losing money left and right while simultaneously inventing new ways to make a social media platform suck to use.
Oh, there’s scope to introduce “official” Reddit user badges for 10 bucks a pop and all of sorts of suicidal Musk shenanigans
Wait until you have to be logged in to read any posts on Reddit.
Didn’t we achieve this by turning all subreddit NSFW?
deleted by creator
Yes, but when the users and mods who built the site do it, it’s petty and selfish. When the ceo does it, it’s brilliant, compounds user value, and improves the security of the user experience. /s
I’m down, Spez. What’s bad for Reddit is good for us.
Honestly… We’re all (or a massive chunk of us are) Reddit refugees. I don’t give many fucks about that place. If it thrives, whatever. It’s out of my control. I’m not going to hate post here with fantasies of spez seeing it and being like, “WHYYYYY?”
What little I’ve seen here seems like a disproportionate amount of passionate and quality users came over here. Users that were likely a bit more than casual; ie we we invested enough to use 3rd party clients, some of us are probably former mods who got disillusioned, etc.
We’re all giddy and it is the honeymoon phase and I HOPE HOPE HOPE the users’ enthusiasm doesn’t fizzle like Mastodon seems to but I would say, just use this. Enjoy it. Post worthwhile things. I don’t think Reddit is necessarily going away, but if it’s just some weird bot riddled, low effort posts sort of place, why would we want to be there anyway?
The fediverse will probably last as long as the internet does, if the BBS is anything to go by. And I’ve seen people starting to realize federation is an actually decent option at the very least.
I saw people unironically saying this and being upvoted for it in hackernews, completely turned me off from the site lol
Very good. An evil corporation doing evil things to an evil agenda.
I mean, is it evil? Seems like removing results that can’t be seen without a login might make sense. Google has plenty of faults but I don’t know if this is one of them
Did Pinterest used to require logins to see their images? I remember I used to get annoyed when search results directed me to that site
They absolutely did (do? I never visit it)
If it is just a revenge for Elon not paying fees for the Google hosting, it would be very evil indeed. Of course, from Lemmy point of view, it is just reason to get more popcorn.
I gotta be careful about my sodium intake from all the popcorn.
Not providing a service for free to another company is evil?
The details are unclear as to why these links were removed. Opining on Alphabet’s morals at this point is premature.
It’s not premature because calling Google evil has absolutely nothing to do with this action.
We don’t need to make up reasons to call corpos such as Alphabet evil. The fact of the matter is that at this time, nobody even knows if Google lifted a finger to forward the delisting of Tweets. For all we know, it could be entirely on Twitter’s end.
Hence, assigning blame to Google for this without understanding their role in it is premature.
It could also be the search algorithm reacting to all the users returning to google after seeing only the login page
Why are you so invested in bootlicking for google? They are precisely one of the mega corporations that fediverse exists to get away from. Peertube and other projects target platforms they own just like lemmy targets reddit and mastodon targets twitter. They are one of many centralised ridiculously oversized evil corps that this entire platform aims to subvert.
No bootlicking detected. The first sentence of my comment actually belies my contempt for them.
Not jumping to conclusions is not bootlicking.
He also said “doing evil things” but I’ve seen no evidence that they have “done” anything. This could be just be the algorithm responding to these tweets no longer being visible without login.
Funny Google’s motto is ‘don’t do evil’
It used to be. They removed that.
Well at least nobody can call them hypocrites.
Remember when Google separated their parent services into a new company called Alphabet? Alphabet’s motto isn’t ‘don’t be evil’.
deleted by creator
Elon’s goal is to destroy twitter, per his wife.
You’re gonna have to be more specific. Which wife?
Wait what
Probably some sort of rich person scheme that he can do whereby if his multi-billion dollar mistake ends up being a failure he can write it off and no longer be on the hook for it or something.
Like bankruptcy, but only for twitter and not himself.
He can’t write off 44 billion though, he would get a tiny fraction of that and there are debts to pay back to investors.
I don’t think he’s interested ingetting it back, just to no longer be on the hook for it. I’m sure there’s no doubt various loopholes that he can use to wiggle out of the full consequences of his stupidity and hubris.
So when you do a write off you still are responsible for that money. It’s not declaring bankruptcy, it’s just lowering your taxable income due to loss. So “write-off” doesn’t really make much sense in this context.
Maybe not the correct ‘legal’ term. But it’s splitting hairs really.
He wants rid of his serious burden. And imagine he thinks that he can do that if he kills the platform. Instead of being on the hook for 44billion with the repayments and interest attributed to that every month, he can cut that down to a fraction.
I’m not saying I have any idea how it works, or even if it’s possible. Mainly that I wouldn’t be surprised if it was.
You can’t just say write off and expect anything to happen
Buying Twitter and tanking it lets him write off a huge amount of money for 2023 and get his taxes back from the previous year.
Also he gets to leverage Tesla stock as the company is growing more fiscally viable… Why does that matter? Why sell before it takes off??
Elon bought Twitter with $12.5bn in loans… Secured with the same value of Tesla stock. He didn’t sell it…
So it’s almost free money. If the stock takes off he can make out like a bandit, pay the loan off with 1/2 the value of the real stock once it rises, but despite making money can still claim an overall loss on Twitter…
This is why he doesn’t give a fark if Twitter lives or dies. If Twitter dies it’s a write off, but because half the investment is literally a loan against Tesla stock he still treats it like free money…
There’s a cap on loss deductions, billionaires don’t care about deducting losses. Also he didn’t buy it with Tesla stock, he put up Tesla stock as collateral.
Right…stock as collateral in a loan.
It’s amazing what bullshit Billionaires can do.
Also those losses are per corporate entity so he gets to use those caps multiple times on each corporation and on himself.
In that case let’s thank him for spending few billions doing that! Thank you Elon :p
Can someone please ELI5 what his end game is? I refuse to pay attention to twitter, but Elon, he’s crazy.
You are assuming he has one. Before he overpaid for Twitter, he probably thought he would turn it into a successful version of TruthSocial. Then reality came knocking. Now, he may have no idea what he’s going to do.
When you were a kid, did you ever take apart an old vacuum cleaner or power tool, thinking “I’m going to make something great out of this”? Then, when you had all the pieces lying around you, you wondered why you even started because you have no idea what you are going to do now? That may be Elon at the moment. He’s looking at the parts of his mom’s treasured vacuum cleaner, and wondering how he can blame it on the brother his parents never gave him.
That’s spot on.
This pretty much sums it up for twitter I guess. Thank god we have fediverse (at least for those savvy enough) to partake in
Beautiful analogy.
I can so relate to this, and it honestly does make perfect sense.
When I was 8, I took a hacksaw to my cousins bike and ripped it in half. My plan was to make a unicycle… I only made people upset.
Hubris?
Debris
He does not have one. He’s just high on whatever it is he’s taking.
I feel like Google is going to have to find a way to effectively index federated content at some point. The only way to really get human information is from sites like Reddit and Twitter. And both of those platforms seem to be dedicated to completely imploding at the moment.
There’s nothing about the content being federated that makes it hard or impossible to index. Each instance is just a website with a public webpage that a bot can read. That all a search engine needs to index it. The worst case scenario is the bot will find the same content on multiple instances.
I did read that the website is loaded entirely through JavaScript and that maybe the Google bot doesn’t execute JavaScript so can’t see the text. I don’t know if that’s still a problem in 2023, though.
This article says it’s not a problem, but I didn’t read past the tl;dr, so maybe there’s a caveat. Like maybe it has to use a popular framework like React or something to work.
https://searchengineland.com/tested-googlebot-crawls-javascript-heres-learned-220157
Googlebot does execute Javascript, but since rendering JS needs much more resources, JS crawling will happen significantly less then simple http crawling. That’s why all big sites still return server side rendered content.
Rendering with JS definitely makes a difference, it’s part of the reason SSR is such a big deal for SEO.
It’s already indexing it.
duckduckgo (who uses Microsoft’s index I believe) is able to find Lemmy instances already.
problem is since every instance has its own domain you cannot search all of Lemmy or the more obscure fediverse.
lemmy.world,beehaw.org,programming.devare all different “websites”.I append “reddit” to my query when I want to search reddit for a human answer to a question. Can’t do that with Lemmy, unless the instance is branded as Lemmy.
Unless there will be an org or volunteers that indexes federated instances and makes them available to search engines to they can be differentiated, finding stuff in the fediverse might be difficult…
Would be lovely if we could just start a search with
fedi:oractivityPub:
Fuck Google, if Lemmy continues to take off we can just develop better search tools within the fediverse. The wider internet has been colonized, the path forward cannot rely on big tech corporations.
I’m not a programmer/developer so I don’t even understand the scale of the work that has yet to be done. But I am deeply committed to upsetting the status quo, and this platform feels distinctly revolutionary. Can’t wait to see what the future holds for Lemmy.
It’s all well and good to have a revolution, but if nobody knows you’re having one then nothing really changes. There are still benefits to centralised services, one of which being scale. To effectively index so much data you need scale, which is why smaller search engines tend to be just white labels of things like Bing.
100k people isn’t nobody. Centralized services can be useful at times, but there is no fundamental law preventing a decentralized system from providing the same functionality.
The value of indexing data drops drastically when much of that data is junk, as is the case in the wider internet. Because Lemmy is a federation, there is a built in system to filter the junk.
Ask and ye shall receive. Just saw this post!
I want a federated wiki. But bad actors resistant somehow.
Lol nice. The pace of development of Lemmy is unreal. I’ve only been here a month and it’s already so much easier to use than in the beginning.
Isn’t it automatically indexed? I mean, I can go to lemmy.world in a browser and see the content, wouldn’t Google’s indexing bots do the same?
It already is.
Just put ‘site:lemmy.world’ into Google to see what it has indexed on that instance for example. I don’t think Lemmy is optimised for search yet, but I saw some GitHub threads around the topic.
Isn’t it automatically indexed? I mean, I can go to lemmy.world in a browser and see the content, wouldn’t Google’s indexing bots do the same?
You can tell from how many upvotes this has that there are just as many idiots here as on reddit lol
Google can definitely index Lemmy
Honestly, and I hate this, but I doubt they will. The majority of people will never go federated, even though it’s so easy, because they suck.
It’s the difference between a mom and pop restaurant and McDonald’s.
We don’t need everybody to go to the mom and pop restaurant. Just enough of us to keep it afloat.
People at large really need to remember not every kind of growth is good: it has to be sustainable, and only happen until where it’s needed.
Unlimited growth is basically cancer, and that’s what big corpos feel to society tbh.
deleted by creator
The Porndex says otherwise
That sleeping giant has woken up. And it has morning wood.
yes
Can I please get this guy’s face off my front page? ._.
Wait… so google can just do that? Guess Pinterest bought the premium cloud services and never misses a payment. And tips.
Probably it’s just that the Google crawlers are not being served Twitter content, since Twitter now requires a logged in account. If so the number of indexed tweets should keep dwindling as they expire from the Google index, I imagine. We’ll see.
Yeah, I don’t know what else they’re gonna do. People are already getting pretty fed up with the search results from their core product, the bitching started years ago and it’s starting to get pretty loud, with power users of search becoming determined to find a better solution when they can. Don’t think Google doesn’t hear such things, or isn’t aware of them.
So if search starts returning a bunch of results that are supposed to be Tweets but people keep hitting login paywalls, that just makes Google look bad. There’s already widely circulated plugins - which Google knows about because they’re getting installed on Chromium browsers - to automatically block Pinterest results, people were getting fed up with “fake” image results that weren’t usable. The value proposition of what is still their primary business is under attack and can’t stand more erosion of trust.
Maybe it looks like retaliation for Musk not paying his bills, but it probably isn’t, it’s just more fallout from Musk’s actions.
The Google search result issue came to a head because of the reddit debacle. With the protest sending so many core subs private it immediately killed any seo those subs had on google search. Even Google made the announcement acknowledging that the reddit protest had ruined search results with most of one of the largest and most visited websites suddenly not being searchable.
If I had to hazard a guess I’d say the trouble at reddit is the first thing to put fire under Googles feet in a long time and now the tech giant’s eye is focused back on their main product for generating ad revenue.
So with Elon putting limits on accessibility for his site there is zero chance of Google being able to properly index and cache twitter content, I’d even bet most of the moderation was done by automated indexing systems that could no longer access content.
Do you have a sauce for that Google announcement about Reddit? I’m interested!
Login screens never stopped google from returning useless pinterest spam.
This
I mean, those links are good as dead if you can’t see anything without a login. Pinterest shows you a few things before nagging you to log in.
They do that for any site that isnt serving content anymore…i.e. essentially down. Will come back within a few days usually.
Just Google “NASA twitter” and click on one of the pretty links up top. Those haven’t been removed yet.
People with power shouldn’t be allowed any control over platforms that give power to “the people”. I believe this is all intentional. If it was just Twitter, or just Reddit, or just 1 TV network, I wouldn’t think that but they’re sabotaging all the mainstream platforms at once and making it incredibly obvious. They’re trying to collapse our means of communication ahead of 2024.
Doesn’t sound like retaliation to me, it sounds like their scheduled web crawlers are finding that content they used to index is now no longer viewable and this removed from search results. Pretty standard. My guess is that there were 400 million URLs listed and as the crawler uncovers that they are no longer available, that number will keep dropping to reflect only content publicly viewable. If only 500 URLs are now publicly viewable (without logins) then that’s what they will index. Google isn’t a search engine for private companies (unless you pay for the service) they are a public search engine so they make an effort to ensure that only public information is indexed. Some folk game the system (like the old expertsexchange.com) but sooner or later google drops the hammer.
God, I hated expert sexchange so much. It was a blessing when stack overflow started.
I’m only an amateur but I’m happy to give it a go
a part of me wants this to get worse so that people finally realize elon is insane once and for all
