ECHO — Technical Architecture | Amazon Nova Hackathon

System Overview

How ECHO Works

ECHO is a hands-free voice AI assistant for field service workers. It uses Amazon Nova Sonic's bidirectional streaming to provide sub-second voice interactions with live tool calling — enabling workers to query their CRM, look up chemical safety data, and generate service reports entirely by voice.

🎙️

Browser Mic

16kHz PCM audio capture

→

🔌

Socket.IO

WebSocket streaming

→

🖥️

Node.js Server

AWS EC2 bridge

→

⚡

Nova Sonic

AWS Bedrock streaming

→

🤖

Tool Calls

CRM, Labels, Weather

→

🔊

Voice Response

24kHz audio output

Core AI

Amazon Nova Sonic Integration

ECHO uses Amazon Nova Sonic via the AWS Bedrock Runtime API with bidirectional streaming. This enables true speech-to-speech AI — the model processes audio directly without an intermediate transcription step, resulting in faster, more natural conversations.

// Initialize bidirectional stream with Nova Sonic
const command = new InvokeModelWithBidirectionalStreamCommand({
  modelId: "amazon.nova-sonic-v1:0",
  body: JSON.stringify({
    inferenceConfig: {
      maxTokens: 1024,
      topP: 0.9,
      temperature: 0.7
    },
    audioOutputConfig: {
      sampleRateHertz: 24000,
      encoding: "pcm"
    }
  })
});

const response = await bedrockClient.send(command);

// Process bidirectional audio stream
for await (const event of response.body) {
  if (event.audioOutput) {
    // Stream audio back to client
    socket.emit('audioOutput', event.audioOutput);
  }
  if (event.toolUse) {
    // Execute tool and inject result
    const result = await executeTool(event.toolUse);
    stream.write({ toolResult: result });
  }
}
      

Specification	Value
Model ID	`amazon.nova-sonic-v1:0`
Input Audio	`16kHz PCM, 16-bit signed int`
Output Audio	`24kHz PCM, 16-bit signed int`
Streaming	`InvokeModelWithBidirectionalStreamCommand`
Region	`us-east-1`
Latency	`<1 second` first response

Agentic Capabilities

Tool Definitions

ECHO defines tools that Nova Sonic can call mid-conversation. When the model decides to use a tool, execution happens server-side and results are injected back into the stream — the user hears a natural voice response with live data.

📋

FieldWork CRM

Query today's schedule, search customers by name/address, retrieve service history, and submit documentation reports directly to the CRM via REST API.

Live API

☣️

Chemical Label Lookup

Fetch EPA pesticide labels and Safety Data Sheets from labelsds.com. ECHO reads the actual document to answer compliance questions — zero hallucination.

Zero Hallucination

💾

Submit Note

Write structured service reports to FieldWork CRM customer records. One voice command generates documentation and submits it to the system of record.

Write Access

🌤️

Weather

Real-time weather conditions for treatment timing decisions. Temperature, wind speed, and precipitation affect pesticide application effectiveness.

Utility

🕐

Date & Time

Current date and time for scheduling context. Workers often need to reference "today" or calculate follow-up dates while speaking.

Utility

📅

Schedule Lookup

Pull the technician's schedule for today from FieldWork CRM. Shows customer names, addresses, service types, and time windows.

Live API

Client Architecture

Browser Audio Pipeline

The web client uses the Web Audio API to capture microphone input, convert to PCM format, and stream to the server via Socket.IO. Audio responses are decoded and played back in real-time with seamless scheduling to prevent gaps.

// Capture mic audio at 16kHz
const stream = await navigator.mediaDevices.getUserMedia({
  audio: {
    echoCancellation: true,
    noiseSuppression: true,
    autoGainControl: true
  }
});

const audioCtx = new AudioContext({ sampleRate: 16000 });
const source = audioCtx.createMediaStreamSource(stream);
const processor = audioCtx.createScriptProcessor(512, 1, 1);

processor.onaudioprocess = (e) => {
  const input = e.inputBuffer.getChannelData(0);
  const pcm = new Int16Array(input.length);
  
  // Convert Float32 to Int16 PCM
  for (let i = 0; i < input.length; i++) {
    pcm[i] = Math.max(-1, Math.min(1, input[i])) * 0x7fff;
  }
  
  // Stream to server
  socket.emit('audioInput', toBase64(pcm.buffer));
};
      

Technology Stack

Full Stack Overview

⚡

Amazon Nova Sonic

Speech-to-speech foundation model on AWS Bedrock. Bidirectional streaming with native tool use support.

AWS Bedrock

☁️

AWS Bedrock Runtime

Managed inference API for Nova Sonic. Handles authentication, rate limiting, and streaming connections.

AWS

🖥️

Node.js + Socket.IO

WebSocket server bridging browser audio to Bedrock. Handles session management and tool execution.

Backend

🌐

Web Audio API

Browser-native audio capture with echo cancellation. Float32 to Int16 PCM conversion for Nova Sonic.

Frontend

🔌

FieldWork REST API

Live integration with FieldWork HQ CRM. OAuth authentication with read/write access to customer data.

Integration

📄

LabelSDS API

EPA pesticide label and SDS document retrieval. Enables compliance-accurate chemical information lookup.

Integration

Real World Impact

Field Service Automation

Field service workers spend an average of 2.4 hours per day on administrative tasks — paperwork that takes them away from actual service delivery. ECHO eliminates this friction by enabling voice-driven documentation, CRM queries, and compliance lookups.

Metric	Before ECHO	With ECHO
Service Report Time	15-20 minutes	2 minutes (voice)
Chemical Label Lookup	5+ minutes (phone/manual)	10 seconds (voice)
Schedule Check	Open app, navigate, scroll	"What's my next stop?"
Customer History	Multiple screens, search	"What did we do last time?"
Hands Required	Both (typing)	Zero (voice-only)

Submission Details

Amazon Nova AI Hackathon

🏆

Focus Area

Agentic AI — Agents using Nova reasoning to tackle real-world problems with tool use.

👤

Submitter

FieldMatrix.ai — AI-powered tools for field service professionals.

🔗

Links

Live Demo · Source Code · GitHub

Technical Architecture