Real-time emotion recognition isn’t one model, it’s a sensor-fusion problem. A production system reads a face, listens to the voice, and reconciles both against context: who the user is, what they’re doing, what just happened on screen. In 2026 the accuracy gap between vendors is small. The gaps that decide your product are latency under 300 ms, on-device privacy, honest calibration, and, new this year, which APIs are still going to exist next quarter.
We’ve shipped emotion-aware features in video, telehealth, and ed-tech products, and we’ve replaced most of the tools on this list at least once in production. This guide compares the emotion recognition software our teams actually deploy, with the 2026 pricing, the regulatory traps, and the failure modes we’ve paid for. For the customer-service angle specifically, see our guide to video emotion analysis for customer service.
Key takeaways
• Accuracy is close to a commodity, but only in the lab. Curated sets hit 90%+ (posed CK+, cleaned-up RAF-DB); in-the-wild AffectNet drops to ~65% categorical. Quote the honest number.
• Hume is leaving the API business. Its Expression Measurement API closed on 14 June 2026, and its EVI and TTS APIs end on 13 November 2026 (Hume changelog, 2 October 2026). Move voice emotion to devAIce or your own model now.
• Face-only is a trap. Face + voice + context fusion cuts the confident-and-wrong errors that generate complaints — the failure mode that matters.
• Build-vs-buy flips around 40–60K sessions/month. Below that, an SDK or on-device tool wins; above it, a fine-tuned open model on your own hardware wins on cost and latency.
• The EU AI Act already bans it at work and school. In force since 2 Feb 2025, with fines up to €35M or 7% of turnover. Since 2 Aug 2026 you must also tell people when emotion recognition is on. Check your use case before you write code.
Updated October 2026: Hume’s EVI and TTS APIs shut down on 13 November 2026, so we added migration paths and audEERING devAIce, re-checked MorphCast, Noldus and market figures, and added the EU AI Omnibus dates.
Why Fora Soft wrote this guide
Fora Soft builds video, real-time communication, and AI products, 250+ of them since 2005, with a 50+ person in-house team. Emotion and affect features keep showing up in that work: an attention indicator for a live class, a “how’s this session going” signal for a therapist, a reaction meter for a shopping stream. We’ve wired in commercial SDKs, ripped them out when a vendor deprecated a feature, and moved workloads on-device when the per-session cloud bill crossed the line. Most of it ships through our computer vision development services, where face and expression models run on live video around the clock.
That production scar tissue is the point of this guide. On our computer-vision and surveillance work (see the VALT platform, used by 770+ US organizations) we learned that a model’s benchmark score and its behavior on a real webcam feed are two different things. So this isn’t a feature-table listicle. It tells you which emotion recognition software to pick for your load, your regulatory regime, and your team’s ML maturity, and what tends to break in month four.
Figure 1. A production emotion pipeline: two model streams plus context, reconciled in a fusion layer, output as an affect state, not raw scores.
Weighing emotion AI for your product?
Book 30 minutes with a product-ML lead. We’ll pressure-test your use case, your latency budget, and your compliance surface — no slideware.
What real-time emotion recognition actually measures
Real-time emotion recognition software infers a person’s emotional state from live signals, facial muscle movements, voice tone (prosody), word choice, and sometimes physiology like heart rate, and returns a score within a few hundred milliseconds. Modern systems don’t report “happy” as a hard label. They report continuous valence (how positive or negative) and arousal (how activated), plus derived signals like engagement or attention.
Here’s the catch that trips up first-time buyers: the six “basic” emotions (happy, sad, angry, fear, surprise, disgust) come from Paul Ekman’s 1970s work, and that categorization doesn’t hold up cleanly across cultures. A polite smile in Osaka and a delighted grin in Ohio look different to a model trained mostly on Western faces. That’s why the research consensus moved to dimensional valence-arousal, and why the vendors worth paying for expose it.
Real-time matters because the value is in the loop: a therapist watching a live engagement bar, an agent adapting its tone mid-sentence, a class dashboard flagging a room going quiet. Batch analysis after the fact is a different, easier problem. Everything below is graded on live performance. If you want the engineering foundations behind running models on live video, our AI for video engineering track goes deeper.
Reach for valence-arousal when: you need a trend over a session (“engagement dropped after minute 12”) rather than a per-frame label. It’s more stable, more defensible, and less creepy to show users.
How accurate is emotion recognition in 2026
It depends entirely on which dataset a vendor quotes. On curated sets the best models score high: above 97% on posed CK+ and about 90% on RAF-DB, which is collected in the wild but cleaned and balanced. On AffectNet, faces scraped “in the wild,” the closest thing to your real users — the same class of model lands around 63–70% on 7–8 categories. For dimensional output, 2024–25 state-of-the-art on AffectNet is roughly 0.72 CCC for valence and 0.65 for arousal (CCC of 1.0 would mean perfect agreement with human raters).
So when a sales deck shows you 95%, ask which dataset. The honest production expectation for a single-modality face model on live video is closer to two-thirds correct, useful for aggregate trends, not for judging one person in one frame.
Figure 2. The number a vendor quotes depends on the dataset. Posed lab faces flatter everyone; AffectNet is the honest bar.
Why multimodal beats face-only
We rebuilt one client’s emotion feature three times because the face-only signal kept failing on live calls. Lighting shifts, a hand over the mouth, an off-angle laptop camera, someone eating — all of it wrecks face-only accuracy. Voice is sturdier but loses nuance over telephony codecs. Fuse the two and you don’t just gain a few points; you change the failure mode. In our internal test across five product verticals, fusion cut the confident-and-wrong errors by 35–50% versus face-only. Those are the errors that generate support tickets, because the system isn’t silent, it’s wrong with a straight face. Fusion makes it abstain instead, which users forgive.
Reach for voice over face when: you can only afford one modality and the user talks. Prosody survives bad lighting and cheap webcams. The exception is silent attention monitoring, where face is the only signal you have.
What changed in the 2026 emotion AI market
Market size estimates are all over the map because “emotion AI” means different things to different analysts. MarketsandMarkets sizes the Emotion Detection & Recognition market at $29.14B in 2026, growing to $43.29B by 2031 at an 8.2% CAGR (report published June 2026); other research firms publish figures more than twice that, and broader “affective computing” reports run higher still. Treat any single headline number with suspicion. What matters more is what shifted on the ground in the last 18 months:
1. Hume is winding down its public APIs. In January 2026 Google DeepMind hired Hume CEO Alan Cowen and about seven engineers under a non-exclusive licensing deal. Hume then closed the standalone Expression Measurement API (last new Playground jobs on 14 May 2026, API and downloads off on 14 June 2026). On 2 October 2026 it announced that the EVI and TTS APIs end on 13 November 2026, with account data deleted after that date. If your product depends on Hume for voice emotion, you have weeks, not quarters.
2. Affectiva is a Smart Eye product line. Smart Eye bought Affectiva in 2021 (~$73.5M) and iMotions the same year, and the group now leans toward automotive interior sensing. The upside is a bigger dataset (17.4M face videos and 8B facial frames from 90 countries, by Affectiva’s own count). The downside is that emotion-as-a-product SDK licensing got more enterprise-shaped.
3. Regulation caught up. The EU AI Act’s ban on emotion recognition at work and school has been enforceable since February 2025. That single clause has quietly killed several “sentiment analytics for meetings” roadmaps aimed at European buyers. The EU AI Omnibus (Regulation (EU) 2026/1744, in force since 27 July 2026) left the ban untouched and moved the high-risk obligations for other emotion-recognition systems to 2 December 2027.
4. Familiar names left emotion behind. Realeyes spun its advertising business off as Adverteyes in November 2025 and now sells face-based identity verification. Kairos’s public plans cover identity verification, liveness and face recognition, with no emotion product listed. Microsoft went first: it retired the emotion attributes in Azure Face in 2022 for new customers and by 30 June 2023 for existing ones.
Emotion AI companies still selling a product you can build on (October 2026): Smart Eye (Affectiva and iMotions), MorphCast, Noldus, audEERING, Adverteyes for ad testing, and the open-source stack. Hume stays usable until 13 November 2026.
1. Hume AI — EVI, shutting down 13 November 2026
Hume rebuilt the category around “affect states” instead of basic emotions: dozens of facial expressions, vocal bursts (laughter, sighs, gasps) and speech prosody dimensions. Its EVI (Empathic Voice Interface) is a real-time speech-to-speech model that returns the user’s vocal expression measures with each transcribed message and answers in a matching tone, with round-trips in the 300–500 ms range on a warm connection. EVI 3 and EVI 4-mini (announced October 2025) are the current versions.
Status and pricing (October 2026): Hume’s changelog says the EVI and TTS APIs stay fully supported until 13 November 2026 at 12:01 a.m. EST and then end. Until then EVI costs $0.04–0.07 per minute depending on plan ($0.04 on Business, $0.07 on Starter and Creator). The standalone Expression Measurement API, which once priced video+audio scoring at about $0.083/min, closed on 14 June 2026.
Where to move. For voice emotion scores that stay on the device, use audEERING devAIce (#7) or the open-source emotion2vec+ model (#8). For a voice agent that should sound empathetic, pair a speech-to-speech model with a separate prosody classifier. Our LiveKit AI agent development team builds that split, so the next vendor exit swaps one component instead of the product.
Reach for Hume when: you’re running a prototype or study that finishes before 13 November 2026. Where it breaks: anything that must run after that date. Don’t start a production build on it.
2. Affectiva (Smart Eye) — Affdex SDK
The pioneer, born from the MIT Media Lab in 2009 and now fully part of Smart Eye (acquisition completed 2021, ~$73.5M). Affdex is the one of the most widely deployed facial emotion SDKs, and its data, by Affectiva’s own count 17.4M face videos and 8B facial frames from 90 countries, is the biggest reason to consider it: emotion models fail cross-culturally when trained only on Western faces, and Affectiva has the broadest coverage to push back on that.
Pricing (October 2026): Affectiva publishes no price list. The Emotion SDK is licensed by quote, and automotive in-cabin deals are enterprise contracts. It runs on-device with sub-100 ms inference, which is what makes it privacy-friendly and low-latency at once.
Reach for Affdex when: you need on-device face analysis, a native SDK, and cross-cultural accuracy. Where it breaks: it’s face-only, so pair it with a voice model for anything conversational.
3. MorphCast — browser-native, fully on-device
MorphCast does one thing that nobody else does as cleanly: it runs entirely in the browser. The SDK is under 1 MB of JavaScript and processes every frame in the browser, with no server in the loop. For a GDPR-sensitive product that’s a compliance story you can put in front of a DPO without flinching. It reads over 100 facial indicators and returns the six Ekman emotions, valence and arousal on Russell’s circumplex, attention and engagement.
Pricing (October 2026): plans are sold by stream hours: Minimal $199/month for 50 hours, Plus $499 for 150, Pro $999 for 350, and Enterprise by quote, each with a 60-day free trial. That’s $2.85–3.98 per stream hour at list. There’s no cloud GPU bill, but the licence still scales with usage.
Reach for MorphCast when: you have a web product, a hard privacy requirement, and you want facial engagement without standing up any infrastructure. Where it breaks: it’s face-only and web-first, with no voice channel.
4. iMotions — research-grade biometric platform
iMotions isn’t an SDK you embed — it’s a research platform. It fuses facial coding, eye tracking (Tobii, Smart Eye), galvanic skin response, ECG, EEG, and survey data onto one timeline. University labs, UX agencies, and automotive HMI teams live inside it because it’s the tool that lets you say “the user frowned, their heart rate spiked, and their eyes fixed on the lower-right corner” in one synchronized view.
Pricing (October 2026): quote only. Academic licences are sold per module with an annual renewal, commercial ones as yearly subscriptions, hardware extra. A browser-based iMotions Online covers remote studies. iMotions has been part of Smart Eye since 2021, which is why its eye tracking and Affectiva facial coding sit so close together.
Reach for iMotions when: you’re running formal studies and need to defend results to a client or an ethics board. Where it breaks: it’s overkill, and the wrong shape: for a shipped product feature.
5. Noldus FaceReader
FaceReader is the reference implementation for academic psychology. It classifies the six Ekman emotions plus contempt, neutral, and valence/arousal, and it’s one of the few tools with a dedicated infant model (Baby FaceReader), which makes it a common pick in pediatric and food-sensory research. When a paper needs a citable emotion measurement, it’s usually FaceReader.
Pricing (October 2026): Noldus lists one-time licences at $2,000 (Essential, six basic expressions), $9,000 (Advanced, adds Action Units and Baby FaceReader) and $12,500 (Premium, adds vital signs, voice analysis and webcam eye tracking), each with a year of NoldusCare support. FaceReader Online runs remote panels. Noldus quotes about 99% accuracy on basic expressions, a posed-face lab figure.
Reach for FaceReader when: you need publication-grade, citable Ekman-category output, especially for child studies. Where it breaks: it’s a lab instrument, not a product component.
6. Adverteyes (ex-Realeyes) — video-ad attention and emotion
Realeyes built the best-known opt-in panel for measuring how audiences react to video ads, scene by scene. In November 2025 it spun that advertising business off as an independent company, Adverteyes, which remains Nielsen’s outcome measurement partner. Realeyes itself now sells VerifEye, a face-based identity verification product. Brands and agencies use the ad side to decide which cut of a spot to run. It’s a managed insights service, not a developer SDK — you buy answers, not pixels.
Pricing (October 2026): custom quotes only; neither company publishes a price list.
Reach for Adverteyes when: you test video creative and care about brand lift and second-by-second attention. Where it breaks: you can’t embed it in your own product.
7. audEERING devAIce — on-device voice emotion
audEERING, the company behind OpenSMILE, sells devAIce, an SDK that reads emotion from the voice on the device. Its Expression module returns a dimensional arousal-valence-dominance score and a four-class label (happy, angry, sad, neutral). The SDK ships for Windows, Linux, macOS, Android and iOS on x86-64 and ARMv8, and audEERING says many of its models run in real time on low-power boards such as a Raspberry Pi. After Hume’s exit it’s the most direct commercial replacement for a voice emotion score that never leaves the device.
Pricing (October 2026): licensed by quote. Kairos, which held this slot in earlier versions of this guide, no longer lists emotion detection: its public plans cover identity verification, liveness and face recognition.
Reach for devAIce when: users talk, audio can’t leave the device, and you want a vendor-supported model instead of training your own. Where it breaks: four categories is a coarse palette, so build on the dimensional output.
Stuck between Affdex, devAIce, and building your own?
Our ML team has shipped every option on this list. We’ll map your requirements to the shortest path that actually holds up in production.
8. Open-source stack — MediaPipe + DeepFace + OpenSMILE
By 2026 the open-source emotion stack is a real production option. MediaPipe (Google) extracts facial landmarks at 30+ FPS on a phone. DeepFace gives you a pretrained 7-emotion classifier in a few lines of Python. OpenSMILE (audEERING) pulls the acoustic features that drive voice arousal and valence. For voice, emotion2vec+ is the open model to start from: nine emotion classes, base (~90M parameters) and large (~300M) checkpoints, the large one trained on about 42,500 hours of labelled speech. Fine-tune on AffectNet (~440K manually labelled faces) or FER-2013 and you’ll land within a few points of the commercial leaders on most tasks, and you own the whole pipeline. One catch: AffectNet’s licence is for non-commercial research, so a shipped product needs licensed or its own labelled data.
Cost (2026): MediaPipe, DeepFace and emotion2vec+ are free to use; OpenSMILE is free only for research, and a commercial product needs a paid audEERING licence (or librosa/torchaudio features instead). The real cost is engineering — a senior ML engineer for roughly 6–10 weeks to productionize, plus inference at about $0.08–0.20 per stream hour of analysed video on a cloud GPU, or effectively free on an NVIDIA Jetson Orin Nano (a ~$249 dev kit) at the edge. For how we approach this class of on-device model, see our custom AI video surveillance guide.
Reach for open-source when: you have ML talent, you’re scaling past ~50K sessions/month, or a regulator won’t let faces leave the device. Where it breaks: year-one engineering and ongoing ops are real: this is a build, not a purchase.
2026 emotion recognition software comparison
One table, the columns that decide procurement: what it senses, how fast, what it costs to start, and who it’s for. Prices were checked on vendor pages on 9 October 2026.
| Tool | Modalities | Latency | Start cost (Oct 2026) | Best for |
|---|---|---|---|---|
| Hume (EVI), ends 13 Nov 2026 | Voice + prosody | 300–500 ms | $0.04–0.07/min until shutdown | Prototypes that end before shutdown |
| Affectiva (Smart Eye) | Face, on-device | <100 ms on device | Quote only | Automotive, media, cross-cultural |
| MorphCast | Face, in-browser | Real-time, client-side | $199/mo for 50 stream hours | Web apps, privacy-first |
| iMotions | Multimodal biometric | Real-time + offline | Quote only | Formal research, UX studies |
| Noldus FaceReader | Face (Ekman + infant) | Real-time | $2,000 one-time (Essential) | Academic psychology |
| Adverteyes (ex-Realeyes) | Face + attention | Batch (hours) | Quote per study | Video-ad testing |
| audEERING devAIce | Voice, on-device | Real-time on device | Quote only | Voice emotion that stays local |
| Open-source stack | Face + voice + custom | 30–150 ms on device | Free (infra + build) | Scale, on-device, privacy |
A decision framework in five questions
Answer these in order and the field narrows fast.
1. Is this a product feature or a research study? Studies go to iMotions or Noldus. Product features go to Affdex, MorphCast, devAIce, or your own stack.
2. Do users talk, or are they silent? If they talk, voice buys you the most accuracy: audEERING devAIce on the device, or emotion2vec+ in your own stack. Hume EVI only fits work that ends before 13 November 2026. If they’re silent (attention monitoring), it’s a face job — Affdex or MorphCast.
3. Can faces leave the device? If not, healthcare, minors, strict GDPR: you’re on-device only: MorphCast in the browser, Affdex on native, devAIce for voice, or your own edge model.
4. What’s your monthly volume? Under ~40K sessions, a paid SDK is cheapest. Past ~60K, a fine-tuned open model on your own hardware wins. In between, model both at your real volume.
5. Where are your users? If any are EU employees or students, emotion recognition is likely prohibited for that use, redesign the feature before you pick a vendor. When two answers pull in different directions, the honest move is usually a hybrid — and that’s where a second opinion pays off.
Figure 3. Start from the primary job and follow one branch to a default pick. The same picks, with reasons, are in the table below.
Default pick by primary job (October 2026):
| Primary job | Default pick | Why |
|---|---|---|
| Conversational agent or therapy | devAIce or emotion2vec+ for voice, plus your speech-to-speech stack | Voice carries the signal; Hume’s APIs end 13 Nov 2026 |
| Automotive, media, cross-cultural face | Affectiva (Smart Eye) | Largest face dataset, on-device |
| Browser app, privacy-first | MorphCast | Runs in the browser, no server |
| Formal research with biometrics | iMotions or Noldus FaceReader | Synchronised sensors, citable output |
| Video-ad and creative testing | Adverteyes | Managed panel, scene-by-scene results |
| Over 50K sessions/month or regulated data | Open-source stack | Lowest cost at scale, data stays with you |
| Two jobs at once | Hybrid, e.g. Affdex face + devAIce voice on device | Covers HIPAA teletherapy without a cloud hop |
Table 2. Default pick per primary job, with the reason. Two jobs at once means a hybrid.
Build vs buy — the 2026 unit economics
Here’s the arithmetic we walk clients through. Take a concrete case: 100,000 sessions a month, three minutes each, US-based, no HIPAA. That’s 300,000 minutes of analysis per month.
Cloud per-minute: at the last published Hume video+audio rate of $0.0828/min, that’s 300,000 × $0.0828 = $24,840/month. Even Hume’s cheapest EVI rate ($0.04/min) would be $12,000/month, and both endpoints are gone by 13 November 2026. That’s why teams at this scale stopped treating cloud emotion tagging as the default.
On-device (MorphCast or Affdex): 300,000 minutes is 5,000 stream hours. MorphCast’s largest list plan covers 350 hours for $999, so at this volume you’re on an Enterprise quote, and Affectiva only sells by quote. At MorphCast’s Pro list rate (about $2.85 per hour), 5,000 hours would be roughly $14,300/month before volume discounts. Treat that as the ceiling you negotiate down from; there’s no cloud GPU bill on top.
Open-source self-hosted: figure ~$1,200/month in GPU or edge infra, plus a one-time build of roughly $60–90K. That’s about $7,500/month in year one ($6,200–8,700 across the build range, amortised over 12 months) dropping to ~$1,200/month from year two.
The crossover sits around 40–60K sessions/month. Below it, buy. Above it, and especially if privacy pushes you on-device anyway, the build pays for itself inside a year. When a client takes the build path, our AI integration team uses Agent Engineering to compress that build, so our estimates tend to come in below the industry rule of thumb; when we’re not confident in a number, we’d rather scope it with you than publish it.
| Path | Monthly cost at 100K × 3-min sessions | Status |
|---|---|---|
| Cloud per-minute (last Hume rate, $0.0828/min) | $24,840 | Endpoint closed 14 June 2026 |
| Cloud voice agent (Hume EVI, $0.04/min) | $12,000 | API ends 13 Nov 2026 |
| On-device SDK at list (MorphCast Pro rate) | ~$14,300 | Enterprise quote at this volume |
| Open-source self-hosted, year one | ~$7,500 | $60–90K build over 12 months + ~$1,200 infra |
| Open-source self-hosted, year two on | ~$1,200 | Infra only |
Table 3. At 100K sessions/month the cloud paths are closing or expensive; a negotiated on-device licence and a mature open-source stack are the two real options.
Mini case: a teletherapy platform at 40K MAU
Situation. A US teletherapy product wanted to help therapists review sessions and flag concerning affect patterns. The constraints were strict: a signed HIPAA BAA, under 300 ms latency so the therapist could watch a live engagement bar during the call, and an all-in cost under $0.40 per session. Our healthcare work — platforms like CirrusMED, made the compliance bar familiar.
Plan. The first cut leaned on a cloud face-emotion API for the visual channel. It broke inside a year when that vendor deprecated its emotion labels: the same lesson the whole market just relearned with Hume. So we moved face analysis on-device (HIPAA-friendly, no cloud round-trip), kept voice prosody in the cloud under a signed BAA, and wrote our own engagement-score fusion layer on the client.
Outcome. At 40K monthly active users the stack runs at ~180 ms end-to-end and ~$0.28 per 50-minute session. Therapist satisfaction with the live engagement indicator went from 62% on the old stack to 89%. The lesson we keep proving: the fusion layer, not the vendor logo, was the difference. We now build the same on-device split into our AI telehealth video platform development work from day one. Want a similar assessment of your stack? Book a 30-min stack review.
Privacy, bias, and when NOT to use emotion recognition
Three realities to internalize before you ship. First, the law. The EU AI Act Article 5 prohibits inferring emotions from biometric data in the workplace and in educational institutions, in force since 2 February 2025, with a narrow medical/safety exception and fines up to €35M or 7% of global turnover. “Workplace” is read broadly, so hiring and probation count too.
The rest of the AI Act timeline moved in 2026. Since 2 August 2026, anyone deploying an emotion recognition system must tell the people exposed to it (Article 50(3)). The high-risk obligations for emotion recognition outside the banned settings were due the same day, but the AI Omnibus (Regulation (EU) 2026/1744, in force since 27 July 2026) moved them to 2 December 2027.
Second, GDPR treats emotion data tied to a health inference as special-category data under Article 9. Default to on-device processing, pseudonymization, and opt-in consent that isn’t bundled into your terms of service.
Third, cross-cultural bias is still real. Models trained mostly on FER-2013 (largely Western faces) can drop 8–15 points of accuracy on East Asian and Sub-Saharan faces. Measure bias on your own user population before launch, not after a press cycle.
When NOT to use it at all: when your audience is minors and the value is marginal; when you’d have to show users raw emotion scores (they read as creepy); when a single wrong inference could deny someone a job, a grade, or care; and any EU workplace or classroom use outside the safety exception. Honesty about these limits is part of the product, not a footnote.
Building an emotion-aware feature with a compliance surface?
We’ll review your architecture against HIPAA, GDPR, and the EU AI Act, and tell you what to redesign before you build it.
Five production pitfalls we’ve paid for
1. Treating emotion as discrete categories. Users don’t feel “angry at 87% confidence” — they feel a blend. Use valence-arousal where the SDK supports it; treat discrete labels as a UI simplification, not a model truth.
2. Showing raw scores to end users. Never. Build a derived engagement or mood-trend indicator. Raw confidence numbers feel invasive, and when they’re wrong they’re infuriating.
3. Skipping per-user calibration. Some people have a resting frown; others look perpetually cheerful. Without a baseline you’re measuring face shape, not emotion. Capture a short neutral baseline per user.
4. Ignoring audio codec loss. Narrowband G.711 (8 kHz) keeps pitch and rhythm but cuts the spectral and voice-quality cues above 3.4 kHz that voice-emotion models lean on. Capture at 16 kHz+ with Opus if you control the pipeline.
5. No fallback model. Cloud APIs go down and vendors deprecate features, 2026 was a live demonstration. Keep a second model ready, even a thinner open-source one, so a vendor decision can’t take your feature offline.
What custom emotion recognition software development includes
A custom emotion recognition build has four parts: capture tuned for the signal (16 kHz+ audio, stable face crops), one model per modality, a fusion layer with a per-user baseline and an abstain rule, and a product surface that shows trends instead of raw scores. The face model can be MediaPipe plus a fine-tuned classifier or Affdex; the voice model can be devAIce or emotion2vec+.
Around that sit the parts buyers forget: a bias test on your own users, a consent flow that meets Article 50 of the AI Act, a GDPR and HIPAA review where health is involved, and a fallback model so one vendor decision can’t switch the feature off. Hume’s 2026 exit is the reason we now put that last item in every estimate.
On the open-source path the core ML work is the 6–10 senior-engineer weeks quoted above; plugging an SDK into an existing video app is usually shorter. We scope each build before quoting it, because the fusion rules and the compliance surface vary more between products than the models do.
FAQ
What is emotion detection software?
Emotion detection software, also called emotion recognition software, scores a person’s emotional state from the face, voice or text, usually as valence and arousal plus a few labels. Real-time tools return that score within a few hundred milliseconds of the signal.
What is the best emotion recognition software in 2026?
For on-device face, Affectiva (Smart Eye); for on-device voice, audEERING devAIce; for browser apps, MorphCast; past ~50K sessions a month, a fine-tuned open-source stack. In detail: Affectiva (Smart Eye) leads on-device and cross-cultural face analysis, MorphCast leads privacy-first browser apps, audEERING devAIce leads on-device voice now that Hume’s APIs end on 13 November 2026, and a fine-tuned open-source stack wins at scale. iMotions and Noldus own research.
Is emotion recognition accurate enough for production?
Yes for aggregate signals like engagement and valence/arousal trends over a session. No for single-frame discrete labels, in-the-wild categorical accuracy is around 65%. Treat it as a directional input, never as ground truth about one moment.
Can I legally use emotion AI in EU schools or workplaces?
Generally no. The EU AI Act has prohibited inferring emotions from biometric data at work and in education since 2 February 2025, except for medical or safety reasons. It carries fines up to €35M or 7% of global turnover, so get legal review and a DPIA first.
Is there good open-source emotion recognition software?
Yes. MediaPipe for landmarks, DeepFace for a pretrained 7-emotion classifier, OpenSMILE for voice features and emotion2vec+ for voice emotion form a production-capable stack. Fine-tuned on AffectNet, it lands within a few points of commercial tools — the cost is engineering time, not licenses.
How much does real-time emotion recognition cost at scale?
At 100K three-minute sessions a month (5,000 hours): the last Hume per-minute rate works out to $24,840, an on-device SDK at MorphCast’s list rate to about $14,300 before volume discounts, and a self-hosted open-source stack to ~$7,500 in year one and ~$1,200 after. The build-vs-buy crossover is around 40–60K sessions/month.
Can emotion recognition run on mobile or in the browser?
Yes. MediaPipe plus a LiteRT (formerly TFLite) model runs at 30+ FPS on mid-tier phones, audEERING devAIce runs voice emotion on Android and iOS, and MorphCast runs in the browser with no server in the loop. On-device is the right default for privacy.
Is Hume AI shutting down its API?
Yes. Hume closed the standalone Expression Measurement API on 14 June 2026, and its changelog of 2 October 2026 says the EVI and TTS APIs end on 13 November 2026, with account data deleted afterwards. Move voice emotion to audEERING devAIce or an open model such as emotion2vec+ before then.
Should I build my own emotion model or buy one?
Buy below ~40–60K sessions/month or if you lack in-house ML talent. Build when you’re past that scale, or when privacy and latency rule out cloud. A hybrid of on-device face and on-device voice is often the pragmatic middle.
How big is the emotion detection and recognition market?
MarketsandMarkets puts it at $29.14B in 2026, growing to $43.29B by 2031 at an 8.2% CAGR (June 2026 report). Other firms publish figures more than twice as high because they count different products, so compare definitions before quoting a number.
What is the difference between emotion recognition and sentiment analysis?
Sentiment analysis classifies polarity (positive, negative, neutral), usually from text. Emotion recognition infers a state such as frustration or engagement from the face, voice or body, often as valence and arousal scores in real time.
Which emotion AI companies are still active in 2026?
Smart Eye (Affectiva and iMotions), MorphCast, Noldus, audEERING and Adverteyes sell emotion products you can use in October 2026. Hume’s APIs end on 13 November 2026, Realeyes moved to identity verification, and Kairos no longer lists emotion detection.
How long does custom emotion recognition software development take?
On the open-source path, plan on roughly 6–10 weeks of senior ML engineering for the core models and fusion layer; wiring a commercial SDK into an existing video app is usually shorter. Compliance review and a bias test on your own users run alongside.
What to read next
ML & Vision
7 Best ML Algorithms for Surveillance Anomaly Detection
The vision models next door to affect detection, compared.
Computer Vision
Build Custom AI Video Surveillance with YOLO & DeepSORT
How we ship on-device CV models in production.
AI & Audio
7 Best AI Tools for Audio Apps in 2026
Voice and prosody tooling that pairs with emotion AI.
Services
AI Development at Fora Soft
How we scope and ship AI features end to end.
So which emotion recognition software should you pick?
Accuracy is a near-commodity, so the decision is fought on latency, privacy, modality fusion, cost at your volume, and regulatory fit. audEERING devAIce takes on-device voice now that Hume’s APIs end on 13 November 2026. Affectiva (Smart Eye) takes on-device face and cross-cultural. MorphCast takes privacy-first browser apps. iMotions and Noldus own research; Adverteyes owns ad testing; and the open-source stack is the honest answer past ~50K sessions a month.
The non-obvious call: choose based on where your latency and compliance costs live, not on the accuracy percentage in the deck — and read the roadmap risk, because 2026 proved a vendor can retire a whole API line in one quarter. If you want a second set of eyes that’s shipped all of this, book a 30-minute call with a product-ML lead below.
The book · Volume 6 of 7
AI for Video: A Complete Guide to Intelligence in Video Products: From Models and Inference to Computer Vision, Speech, Search, and Generative Video
If this comparison pushed you toward the open-source path with MediaPipe and your own fusion layer, the next questions are about the models themselves. Volume 6 walks through face detection and tracking, speech recognition and diarization, deployment topologies from device to cloud, and the lifecycle of an AI feature, so your team can judge an affect model on real webcam feeds, not lab benchmarks. Written by Nikolay Sapunov, CEO at Fora Soft.
Read it on Amazon →Kindle & paperback · Fora Soft Video Engineering Handbook
Let’s map your emotion AI stack together
30 minutes with a product-ML lead at Fora Soft, an architecture conversation, not a pitch.
