The dealer at table nine does not blink, because it has no eyelids. What it has is a four-microphone array in its chest and a forty-millisecond inference loop, and by the time you finish saying “raise” it has scored the first eighty milliseconds of the word for pitch instability and decided you are bluffing. You are not bluffing. You have a nine-high straight and a head cold. The machine cannot tell the difference and does not need to, because the house takes its rake either way.
Down the street, a patrol unit stops a Civic with a broken taillight. Before the officer reaches the window, the bodycam has streamed thirty seconds of the driver’s voice to an edge model and returned a deception likelihood of 0.71. The number goes in the incident report. The report goes in a database. Eight months later a prosecutor reads the number and does not read the paragraph explaining what it measures.
A kiosk at the airport asks a woman why she is traveling. She is traveling because her mother is dying. The system records elevated arousal and inconsistent affect and sends her to secondary screening, where a human being will eventually work out what the machine could not. Elsewhere a proctoring tool grades a sixteen-year-old’s attention off her webcam and mails her parents a weekly engagement percentile. A claims system flags a man as evasive because he speaks slowly, which he does because of a stroke.
None of that requires technology that does not exist. All of it requires technology that does not work.
Both problems in that scene were already solved
Start with the poker table, since that is the part everyone finds charming. In a 2017 competition reported in Science the following year, Carnegie Mellon’s Libratus beat four elite heads-up no-limit specialists across 120,000 hands. Two years later Pluribus took on six-player hold’em, beating Darren Elias and Chris Ferguson over 5,000 hands each, then winning a 10,000-hand run against a rotating table of thirteen professionals who had each won more than a million dollars playing the game.
Neither system had a camera. Neither had a microphone. Neither knew what any human at the table looked like, sounded like, or felt. Pluribus computed its blueprint strategy in eight days on 12,400 core hours and played live on twenty-eight CPU cores, which is to say the machine that beat the best poker players alive ran on less compute than the laptop you are reading this on, and it won by treating your emotional state as irrelevant. At a poker table, mostly it is.
The interrogation half of the scene has a different history. It was not solved. It was adopted. The polygraph has been in continuous institutional use for a century despite the 2003 National Research Council review, commissioned by the Department of Energy after the Wen Ho Lee affair, which examined roughly eighty studies and concluded that specific-incident testing discriminates lying from truth-telling above chance but well below perfection, that its false-positive rate is unknown, that most of the underlying research is low quality, and that its accuracy cannot justify federal reliance on it for employee screening. Congress had reached a version of that conclusion fifteen years earlier, when the Employee Polygraph Protection Act of 1988 barred most private employers from using the things at all.
So one problem got solved by math that ignores you completely, and the other never got solved, and the institutions kept the machine anyway. What emotion AI contributed to either story was a price cut. A polygraph exam needs a certified examiner, a room, an hour, and a subject sitting still with wires on. An emotion model needs a WAV file and forty milliseconds. The thing that used to be too expensive to do to everyone is now cheap enough to do to everyone, continuously, without telling them.
What the technology actually is
There are two generations of this, and vendors are careless about which one they are describing.
The first extracts hand-crafted acoustic descriptors: fundamental frequency contour, RMS energy, speaking rate, pause length and distribution, jitter (cycle-to-cycle variation in pitch period), shimmer (cycle-to-cycle variation in amplitude), harmonics-to-noise ratio, mel-frequency cepstral coefficients, spectral tilt. Toolkits like openSMILE will hand you several thousand of these from a single utterance; the eGeMAPS set trims it down to eighty-eight chosen for clinical plausibility. Feed the vector to a support vector machine or a gradient-boosted tree and you have a 2005-era emotion classifier. When a vendor tells you its system extracts fifty-one parameters from the voice and associates them with sixteen emotions, that is the architecture being described, with the arithmetic presented as insight.
The second generation is self-supervised speech transformers. wav2vec 2.0, HuBERT and WavLM, pretrained on tens of thousands of hours of unlabeled audio, then fine-tuned with a small classification head on a labeled emotion corpus, producing either categorical output (angry, happy, sad, neutral) or dimensional output (valence, arousal, dominance). This is real engineering and it genuinely outperforms the older pipeline. It is also the point where the problem stops being about model architecture at all.
Ask where the labels came from. IEMOCAP, the most-cited benchmark in the field, is five sessions of actors performing scripted and improvised scenes in a university lab. CREMA-D is actors. RAVDESS is actors. MELD is clipped from a sitcom. In none of them is the ground truth what the speaker actually felt; it is what a small panel of annotators guessed the speaker felt, picked off a dropdown with the context stripped out. So the model learns to reproduce a crowd’s stereotype of what performed emotion sounds like. That is a real function and a learnable one. It is nowhere near the function on the invoice.
There is a scene in When Harry Met Sally that settles this more efficiently than any benchmark. Sally sits down at Katz’s and performs an orgasm at the table, in a crowded deli, at lunchtime, for the sole purpose of demonstrating that Harry’s confidence in his ability to tell the difference is unearned. She is completely convincing. The woman at the next table places her order accordingly, and that order has outlived most of the rest of the movie.
Run that audio through any system on the market and you would get high arousal, positive valence, and a confidence score in the nineties. Every acoustic measurement would be correct. The one thing anybody in that deli actually wanted to know would be missed entirely, and it would be missed for the best possible reason: the model was trained on people doing precisely what she was doing. Meg Ryan at that table is not an edge case for emotion recognition. She is the training set.
The receipts sit in the cross-corpus tables, which do not make it into sales decks. A 2026 benchmark of speech language models fine-tuned Qwen2-Audio on IEMOCAP and got a macro-F1 of 82.4 on IEMOCAP’s own test set. That same fine-tuned model scored 31.5 on MSP-Podcast and 24.7 on MELD. Train it directly on MSP-Podcast, which is real podcast speech and the closest thing the field has to natural conversation, and the in-domain number is 29.2. Not the transfer score. The in-domain score. Acted emotion is easy, and actual human speech, with context and ambiguity and people talking over each other, sits near a coin flip once you ask for more than two classes.
Arousal is the one signal that travels. Dimensional models trained on MSP-Podcast reach concordance correlations around 0.76 to 0.82 on arousal, while the best published valence prediction without linguistic features sits near 0.64. A microphone can establish that you got louder and faster. It cannot establish whether you got louder because you are angry, excited, embarrassed, hard of hearing, or standing next to an air conditioner.
For a cleaner view of the ceiling, look at a much easier problem. The 2011 INTERSPEECH Speaker State Challenge posed a binary question: drunk or sober. Its benchmark was the Alcohol Language Corpus out of LMU Munich, 162 German speakers recorded both intoxicated and sober, which gives you matched within-speaker pairs, controlled recording conditions, and ground truth from a blood alcohol reading instead of a panel of annotators guessing. Alcohol leaves a documented acoustic signature. Jitter and shimmer rise, the pitch contour destabilizes, voice quality turns breathy and creaky, articulation rate and rhythm shift measurably. Two classes, real physiology, verifiable labels, the same feature stack everyone else is using. The best system in the field got it right 70.5 percent of the time. Hold that against a vendor claiming sixteen emotion categories off uncontrolled audio with annotator guesses standing in for truth. The research community can just about tell drunk from sober; the commercial layer built on top of it is selling contempt versus disgust.
The face side is worse and much better documented. In 2019, Lisa Feldman Barrett, Ralph Adolphs, Stacy Marsella, Aleix Martinez, and Seth Pollak published a sixty-eight-page review in Psychological Science in the Public Interest, the most comprehensive assessment of the evidence anyone has done. It found that particular facial configurations do not reliably signal particular emotion categories, and that people cannot reliably infer emotional state from facial movement. The authors recommended dropping the phrase “emotional expression” in favor of “facial configuration,” on the grounds that the first term assumes the conclusion. It was published the same year the video-interview screening industry hit scale, and made no visible difference to it.
This is not a fringe reading of the literature. When the European Union banned emotion inference in workplaces and schools, Recital 44 of the AI Act named the reasons in plain language: limited reliability, lack of specificity, limited generalisability.
The Nemesysco tell
The Israeli company behind Layered Voice Analysis, the technology at the center of that Digital Trends piece, has been through this argument once already. In 2007 the phoneticians Anders Eriksson and Francisco Lacerda published an analysis in the International Journal of Speech, Language and the Law that worked directly from the patent and concluded the method has no scientific validity, that no theoretical link exists between its waveform measurements and any emotional state, and that the measurements are imprecise enough to be driven by arbitrary artifacts of the recording setup. No software update fixes that, they argued, because the defect is conceptual.
The company’s lawyers wrote to the publisher threatening a UK libel suit, and Equinox pulled the article off its website. Copies were circulating within a week. The suppression is not the interesting part, since it failed almost immediately. The interesting part is that a methods critique drew a libel letter instead of a rebuttal, and that the product is still on sale.
The scary part is not that it works
Consider what a score actually does inside an organization. Nobody buys these systems because they are accurate. They buy them because a number moves responsibility off a person. A hiring manager who rejects you owes someone an explanation. A hiring manager who forwards a candidate confidence score of 41 owes nobody anything. The number functions as an alibi with decimal places, and once it exists it becomes the artifact in the record that everyone downstream treats as evidence.
Then there is the arithmetic, which is worse than the psychology. iBorderCtrl, the EU Horizon 2020 border pilot that tracked thirty-eight facial micro-gestures for what its designers called biomarkers of deceit, reported around 76 percent accuracy from a study of roughly thirty people who had been instructed to lie. Screen a million travelers of whom a few hundred are actually deceptive and a genuinely 76-percent-accurate test throws flags that are overwhelmingly false. Researchers made exactly this Bayesian argument before the pilot ended. No amount of model improvement fixes a base rate; you would need accuracy figures nobody in this field has produced outside a laboratory.
The errors also land unevenly. That iBorderCtrl validation study used thirty-two participants, twice as many men as women, ten of Asian or Arabic descent, and no Black or Hispanic participants at all. Prosody varies with language, dialect, region, age, sex, hearing, medication, and neurotype. A model trained on American actors performing anger will read flat affect as disengagement, a second-language speaker’s processing pause as evasion, a stutter as anxiety, and an autistic candidate’s speech as whatever the training distribution says unfamiliar sounds like. Those failures concentrate on the people with the least standing to argue about them.
The cost people underrate is what the whole arrangement does to the conversation. Once you know a microphone is scoring you, you stop answering the question and start managing a signal. That happens whether or not the model works.
A checkpoint is the worst place to look for nervousness
The founder of Nemesysco has said the original idea was a lie detector for entry gates in Israel, built to stop attacks before they happened. I have stood at those gates. I was nineteen, in uniform, and the thing nobody building a screening system wants to hear is that at a checkpoint the base rate of fear is one hundred percent. Everyone is nervous. The kid on his way to a job interview is nervous. The grandmother with the wrong paperwork is nervous. The soldier checking the paperwork is nervous. Nervousness carried no information there, so we were trained toward documents, stories, and inconsistencies instead of faces and voices. Decades later I build production voice agents for a living, and I can tell you exactly what happens to jitter and shimmer once a voice crosses a narrowband phone codec with automatic gain control somewhere in the path: they stop measuring the speaker and start measuring the network. A feature that fragile in my telemetry has no business in anyone’s employment file.
Now put it on somebody’s face
Every deployment described so far has an institution behind it. A company, a border agency, a school. Institutions can be sued, audited and regulated, which is the reason the European prohibition is scoped to workplaces and educational institutions in the first place. The whole regulatory response rests on that assumption, and consumer wearables walk straight through it.
The hardware already shipped. Meta Ray-Ban Display carries a twelve-megapixel camera, a six-microphone array, and a monocular in-lens screen only the wearer can see, paired with a wrist band reading surface electromyography for silent gesture input. The screenless Gen 2 frames run a five-microphone array. In February 2026 the New York Times reported on internal Meta documents describing a facial recognition feature called Name Tag, which would identify people on sight and surface information about them through the AI assistant. One memo argued for launching during a dynamic political environment in which the civil society groups most likely to object would have their resources committed elsewhere. Wired found dormant code for the feature in the Meta AI app in June. Senator Markey wrote to the company. Meta’s public line is that it is still thinking through options.
Name Tag may never ship, and the shape of the problem does not depend on it. Once a device holds a camera and a microphone array pointed at whoever the wearer is looking at, runs continuous inference, and returns a private result to a display nobody else can see, adding an affect score is a model swap rather than a hardware program. Quantized wav2vec-class models already run on phone-grade silicon. The feature costs almost nothing to build on top of what is sitting on shelves right now.
And that version loses every constraint that makes the hiring case survivable. A rejected candidate at least receives a decision he can contest, in a jurisdiction with a statute, against a company with a legal department and a retention policy. The person on the other side of a pair of glasses gets no notice, no record, no appeal, and no way to know a number was ever generated about him. It never lands in a file where a regulator might find it. It appears in one person’s field of view during a salary negotiation, an apartment viewing, a first date, or a traffic stop, and then it is gone.
Whatever the model gets wrong about flat affect, an unfamiliar accent, a second-language pause or a stutter, it now gets wrong in a setting where the wearer has a confident readout and no way to check it against anything. He just gets the number, and it feels like information.
Where it is actually useful
There is a real version of this technology and it is duller than the pitch. Arousal detection in a contact center works: not “the customer is angry” but “energy and speaking rate rose sharply over ninety seconds, put a supervisor on the line.” The claim is narrow, the acoustic mechanism is real, and the alert ends with a person picking up the call.
Aggregate research signals are fine on the same logic. A media company measuring where arousal dropped across two thousand viewers has a usable hypothesis about its edit and nothing whatsoever about any individual viewer. Cross that line and you have left the range where the model was ever validated.
Clinical vocal biomarkers stand up under protocol, with a clinician reading the output, since Parkinsonian speech and depressive prosody have measurable acoustic correlates and a physiological mechanism behind them. Most of this industry lives in the gap between that research finding and a shipped product feature. Driver drowsiness monitoring has enough behind it that the EU wrote an explicit exception for it into the ban.
Generation is a different problem from inference. Prosody control in text-to-speech and dubbing is a specification task: you tell the model what to sound like. My own voice agents depend on it, and not one of them claims to know how a caller feels. The honest versions all share a shape. They stay on arousal, they describe populations, they hand off to somebody who can be argued with, and they run where the subject can see them.
How to beat it
Start with the law, because it is the strongest lever and most people have no idea they are holding it. In the European Union this is not a gray area. Article 5(1)(f) of the AI Act has prohibited inferring emotions from biometric data in workplaces and educational institutions since 2 February 2025, with narrow medical and safety exceptions, and penalties reaching thirty-five million euros or seven percent of global turnover. It applies based on where your people work, not where your company is incorporated. If your HR stack runs sentiment scoring on employee calls, that is the exposure.
The United States is patchier but not empty. The Illinois Artificial Intelligence Video Interview Act requires notice, an explanation of what the system evaluates, and consent. Massachusetts turned out to have the sharpest instrument. In Baker v. CVS Health, an applicant argued that a HireVue video interview analyzing his facial expressions constituted a lie detector test under a state statute written for machines with wires. Judge Saris found he had standing on an informational injury, reasoning that the missing notice would have primed him to view the interview more critically, and that the statute plainly creates a private right of action. The case survived dismissal, settled in July 2024, and has since seeded copycat suits. The federal Employee Polygraph Protection Act is sitting there too, drafted in 1988 and never repealed.
So the practical move is a written question. Does your assessment process use any system that infers emotion, affect, personality, or truthfulness from my voice, face, or body language, and may I request human review of any automated score? Send that to a recruiter and watch what happens. Most legal departments do not want that answer on paper, and the fastest route to a human reviewer is making the automated one expensive.
If you are already in the chair, understand what the model keys on, which is arousal and channel. Not content, and certainly not truth. Reduce your dynamic range: even volume, even pace, few filled pauses, limited pitch excursion. A narrow prosodic band scores as calm and confident in nearly every implementation of this. The optimal strategy against a machine claiming to detect authenticity is to sound less like a person, which should tell you something about what is being measured. Flattening and performing are the same lever pulled in opposite directions. Nora Ephron established the principle in 1989 and nobody has had to revise it.
Fix your channel, because it may weigh more than your emotional state does. Jitter, shimmer and harmonics-to-noise ratio are sensitive to bitrate, noise floor and automatic gain control, the specific failure Eriksson and Lacerda identified almost twenty years ago. Use a wired headset, a quiet room and a stable connection. You are tuning a larger input variable than your nerves.
Structure your answers. Silence before a response reads as uncertainty in most systems, so front-load your conclusion and do not restate the question. Keep answers in the forty-five to ninety second range, since very short ones score as disengaged and very long ones accumulate variance. Land on a falling pitch contour, because trailing upward reads as hedging.
None of that reaches a stranger’s glasses. Notice requirements attach to employers and schools, and there is nobody on a sidewalk to send a written question to. The one American statute with real reach is the Illinois Biometric Information Privacy Act, which covers voiceprints and face geometry regardless of who collects them and lets individuals sue without proving harm. It is why Meta paid $650 million to settle the Tag Suggestions class action in Illinois and $1.4 billion to Texas under that state’s biometric law. In August 2024 the Illinois legislature capped per-person damages, which took a real bite out of the only lever that was working.
I am not offering any of this because gaming a screening tool is admirable. I am offering it because a system that five paragraphs of advice can defeat had no business making a decision about your life. If speaking evenly and buying a better microphone moves your integrity score, the score was never measuring integrity, and you can test that yourself in an afternoon.
The one at the table is not the one to watch
The robot at the poker table is never going to read you. It is going to compute a bet size from a blueprint strategy on twenty-eight cores and take your money without once wondering what you feel, because at that table the information was worthless. The one at the interview desk claims to read you, on weaker evidence, with greater consequence, and nothing of its own at risk. The one walking toward you on the sidewalk has all of that, plus no name on it and nowhere to send the complaint.
About the Author
Gal Ratner is the founder and CTO of Inverted Software and WhiteStar Labs, and Chief Architect at Prana Entertainment, a Las Vegas enterprise software and AI consultancy. He has spent nearly thirty years shipping production systems on the Microsoft and .NET stack for clients including Microsoft, Sony, Rockstar Games, Best Buy, and Allegiant Air, and now builds production agentic AI: MCP servers, RAG pipelines, SQL Server 2025 vector search, the PLogger observability framework, and voice agents that have to survive real phone codecs before they survive a demo. He is a Los Angeles Business Journal CTO of the Year finalist and the author of the novel The Archive of Lost Suns. He writes at galratner.substack.com as the .NET AI guy that ships.