Articles by Tal
Activity
5K followers
-
Tal Klein posted thisEvery GEO product I've been pitched (and the one I built myself) measures what an LLM tells a stranger. A study that went out last week, The Personalization Gap by Cassie Wilson Clark and João Gabriel Oliveira Silva, tested what happens when the LLM knows who's asking. They ran the same 30 prompts through clean API calls and through accounts carrying a user history. With history in play, Gemini's recommended brand set diverged by 33.3 percentage points beyond normal volatility, and ChatGPTs by 16.0 percentage points. The clean API calls, which are the scalable way to track this stuff, recovered 79% of the brands that kept showing up for the accounts with history. Yes, it's a small study on consumer categories in the UK, so I'm taking it in accordingly, even though it matches my gut. We track ourselves across several models every week, and our tracker asks as a nobody (we do a double-blind sanitation check to avoid bias). The head of EUC I'd like to reach asks their LLM of choice from an account that knows what they run today, what's bothering them, what they feel good about, etc... So, without that context, does it really matter what a LLM says when it's vanilla prompted with, "Which DEX vendor is the most beautiful of them all?" I'm not really sure where we go from here. Ask our customers to ask their LLMs to give us a digital twin head of EUC skill to run our prompts against? Or just roll with what we've got and be happy that we can at least move the needle on the non-divergent responses. Despite what my wife would tell you, sometimes I hate being right.
-
Tal Klein shared thisLakeside customers I speak with often tell me of cross-departmental IT meetings where they spend a lot of time looking at infrastructure dashboards that say everything is fine while the SysTrack employee experience index reports otherwise. They call it the watermelon effect: green on the outside, but red on the inside. I wrote a new piece for CMSWire about how this exact same illusion plagues customer experience. Teams celebrate perfect KPIs and a drop in support tickets, but sometimes that just means their customers are either suffering silently or gave up trying to get help. I broke down why some measurement tools create these blind spots and how we can borrow some principles from site reliability engineering to get an honest picture of what people are actually experiencing. Oh, and if you read it, you'll get to learn all about my latest drama in my role as Klein family CIO, too. 😅 Link to article on CMSWire: https://lnkd.in/gAXF9AcAThe Watermelon Effect: Why Happy Dashboards Hide Unhappy CustomersThe Watermelon Effect: Why Happy Dashboards Hide Unhappy Customers
-
Tal Klein posted thisHey Bradford Stephens, remember when I was saying HTML6 needs an MCP interface? I feel like I'm the new Shingy 🤣 WebMCP, a proposal from Microsoft and Google engineers lets a website expose its content as a skill or "tools" an AI browsing agent can call. The idea is to reuse your existing front-end code, and expose its content in a way a browsing agent can use to help users accomplish tasks on your site! Much of the work I've done on GEO has been about getting cited in the answer. Now, this is about what happens after the citation, when the buyer's agent lands on your site to do something. Today, that agent scrapes and guesses. With a registered tool, I could hand it a function like "compare us against these three vendors for a 15,000-seat estate" and the answer (hopefully) would come back with my math in it. We're already building a public DEX buyers guide out of our panel data, and if this works, I can make that page callable vs. readable. No idea whether buyers' agents will ever use this functionality but if they do, I'll be curious what they ask first.
-
Tal Klein posted thisDouglas Adams keeps turning out to be more and more prophetic. In the first Hitchhiker's book, hyperintelligent beings build Deep Thought to give them the Ultimate Answer. After seven and a half million years of inference it comes back with 42, and when they complain, it points out that they never actually knew what the question was. That's pretty much where I am with AI. Daniel Salinas and I were trying to understand a post he wrote months ago that continues to be insanely engaged with. I fed it to three different LLMs and they all told me why it worked, down to the hook and the timing of the first ten comments. I asked them to write the next one and they all came back with crappy AI slop. The explanation comes easy. A path forward needs the question, and the question is buried in stuff the model can't see… context like who's actually reading and what they scrolled past before they got to you, etc. Adams covered that part too. When Arthur Dent finally pulls the question out of his subconscious in the second book, it turns out to be "what do you get if you multiply six by nine?" (That's 54, not 42, thus predicting AI hallucination). Every AI breakdown of something that gained traction holistically feels like a variation of that to me, a question reverse-engineered from the answer that wouldn't have gotten you there. I've made my peace with that. Figuring out the question is the fun part anyway, and it's the one piece of this job I'd be sad to hand off to code, however deep its thoughts may be.
-
Tal Klein posted thisChatGPT's share of AI prompt volume has cratered: 70% in January, 50% in June (according to Comscore Social (formerly Shareablee, Inc.)'s Q2 AI Intelligence Report which came out earlier this week). Over the same six months Gemini went from 17% to 30% and Claude from 2% to 11%. Comscore's conclusion is that brands "can no longer assume that a single AI experience defines the market." In the parlance of my 13 year old, "Well, duh." Still, most GEO solutions I'm getting pitched are built as ChatGPT programs with the other LLMs as a footnote. If you set your visibility baseline in January, half of the market you were measuring has since moved to LLMs that pull from different sources and cite differently. Heck, I'm even seeing dramatic citation differences between different models withing models (Sonnet vs. Opus produce wildly different citations, for example). We track ourselves across several models every week now, and the spread between them tells a chaotic story. It's a mess out there.
-
Tal Klein posted thisGartner just published a consumer survey with a stat that caught my eye: 65% of consumers believe brands produce too much AI-generated content, and 57% say AI-generated content has made them less trusting of brand messaging (source in the comments). I know it's a consumer shopping survey, so, yes, I'm applying a heap of salt to the B2B relevancy, but those consumers have day jobs (and some of them are the EUC directors who [hopefully] research us). Also! Gartner has now officially entered "AI slop" into their lexicon, which pleases me because I never imagined that would formally happen, and because I'm now expecting to be able to inject "the six-seven of it all" into my inquiries. (Yes, I know LinkedIn got there first. I think it was Colin Steele who first pointed out the "seems like AI slop" button, and already it's been clicked over a million times just in its first two weeks. I'm still inside the parentheses, I know, but I don't know how to get out of them while still being able to point out that posts it classified as slop got 40% fewer views. So the platform I'm posting this on already penalizes what the survey describes.) What I take from all of this is that if Gartner's talking about it then AI slop has officially jumped the shark, and slop volume is now a cost. AI made output nearly free, so we all made more of it and now we've got a classic economics supply and demand problem. Therefore, as Heather Bicknell will attest, the question I've started asking in content reviews is whether a human would have bothered to write this without the tool, because if the answer is no, the buyer can tell and then we're back to square one. Which, honestly, is fine with me. Write less, say more.
-
-
Tal Klein reposted thisTal Klein reposted thisOrganization-wide DEX scores are useful. Role-level insight is powerful. When financial firms connect employee experience data to the work people actually do, they can prioritize the problems that truly affect payments, fraud and customer-facing teams. Our Chief Product & AI Officer Oli Giordimaina explores how in his latest Finextra piece: https://lnkd.in/gjBHaKmb #DigitalEmployeeExperience #FinancialServices #AIOpsHow financial firms can get more value from employee experience data: By Oli GiordimainaHow financial firms can get more value from employee experience data: By Oli Giordimaina
Tal Klein replied to a comment
3h
Aaron Videtto VIDETTO! IN MY OFFICE, STAT!
Tal Klein replied to a comment
3h
Tony White thanks! I tried reaching out to you. Did you get my email?
-
Tal Klein liked thisTal Klein liked thisGood luck NVIDIA. I am sure we will meet again. Current retirement mood - https://lnkd.in/gAKFYYKBCherry Poppin' Daddies - Zoot Suit Riot [Official Video]Cherry Poppin' Daddies - Zoot Suit Riot [Official Video]
Experience & Education
Publications
-
Four Thoughts on IT, SaaS, and Windows 10
Pulse
See publicationEnd-user analytics is the big support gap in end user computing – how can IT support (and ideally risk manage) end-users that are interacting with privileged data on unmanaged devices using unsanctioned applications? They have to make a compact with the users that benefits the users by providing them with support regardless of device, operating system, applications, and behavior.
-
Has Security Ops Outlived Its Purpose?
Dark Reading
See publicationCISOs will need more than higher headcounts and better automation tools to solve today's security problems.
-
The Case for Naked Risk Management
International Association of Privacy Professionals
-
Looking Shadow IT In The Eye, Realizing You’re Staring At Your Own Reflection
Information Security Buzz
-
Mind the Cloud Security Gap
WIRED
See publicationA piece about the root of existing "defense in depth" limits in protecting SaaS usage.
Languages
-
English
Native or bilingual proficiency
-
Hebrew
Native or bilingual proficiency
Recommendations received
14 people have recommended Tal
Join now to viewOther similar profiles
Explore more posts
Explore top content on LinkedIn
Find curated posts and insights for relevant topics all in one place.
View top content