How to Build a SIP and WebRTC Gateway for Voice AI



Build a voice AI gateway in three moves: put your own session border controllers in front of your current vendor, shift phone calls to your gateway in stages behind them, then move browser calls with your own WebRTC SDK. This order gives you routing control and visibility before you replace anything that carries audio.
This is the sequence Retell used to move 100% of its phone and browser calls off a telephony vendor. The full engineering account is here. This article turns it into a plan another team can adapt.
What does a voice AI gateway have to do?
A gateway connects phone networks and browsers to your AI infrastructure. It has to speak two protocols and do six jobs.
| Job | Phone calls | Browser calls |
|---|---|---|
| Signaling | SIP | Your SDK's own signaling |
| Audio transport | RTP | WebRTC |
| Call setup and teardown | Yes | Yes |
| Audio mixing | Yes | Yes |
| Transfers | Yes | Where applicable |
| Supervisor controls (live listening, take-over) | Yes | Yes |
Around the gateway you need two more parts. Session border controllers (SBCs) face carriers and customers' phone systems, receive SIP signaling, and route each call. A WebRTC SDK is the code a website or app uses to place browser calls straight to the gateway. In Retell's design the gateway talks to the AI infrastructure over HTTP and WebSocket.
Should you build one at all?
Only if call quality, availability, and call-level customization are central to your product, and you can staff a real-time communications team permanently. The decision has its own article: should you build your own voice AI gateway. The rest of this page assumes the answer is yes.

Take control of the SIP entry point first
Deploy your own SBCs and point all SIP signaling at them before you write any media code. At first the SBCs forward every call to your existing vendor, so nothing about the call changes.
What you gain right away:
- Routing control. You decide where each call goes, per call.
- Visibility. Every phone call reaches you first, whether or not it connects. Calls that fail upstream stop being invisible.
What it costs:
- A customer-facing change. Customers who send you calls directly need to point at your address and add your IP addresses to their allowlists. Start this early. It runs on your customers' change calendars.
- A temporarily longer path. Each call leaves your network for the vendor and comes back. Retell's calls crossed the public internet twice during this phase.
Build the gateway behind your SBCs
Deploy the gateway inside your own network, behind the SBCs. Now each phone call has two possible paths, vendor or gateway, and you choose per call. In Retell's rollout the SBCs chose the path for inbound calls and the AI infrastructure chose it for outbound calls.
Keeping the SBCs and gateway on the same network pays off in the audio path. Retell measured about 5 ms for the hop between its SBCs and its gateway, against about 50 ms for the public internet hop to the vendor.

Shift phone traffic in stages
Move traffic in small steps and widen only when a step holds. Retell's steps were 1%, 5%, 10%, 20%, 30%, 50%, then 100%.
At each stage, confirm four things match the old path:
| What to preserve | What it means |
|---|---|
| Protocol behavior | Carriers and customers' phone systems get the SIP behavior they expect |
| Audio timing | No new gaps or delays in either direction |
| Participant permissions | A supervisor using live listening keeps the same rights |
| Call lifecycle semantics | Calls start, transfer, and end the same way and send the same events |
The fourth row is the easiest to miss. Your customers' integrations depend on the events a call emits. A call that works but sends a different event sequence is a breaking change for them.
Keep the per-call routing switch until well after 100%. It is your rollback.
Move browser calls with your own WebRTC SDK
Browser calls come last because they are a separate integration, not a variant of phone calls. They skip the SBCs and connect to the gateway directly, so they need their own:
- SDK, shipped to and adopted by your customers
- Authentication
- Connection setup
- Audio transport
Budget for the adoption curve. Phone calls move when you change a routing rule. Browser calls move when customers upgrade an SDK. Retell's is documented in its web call guide.
When browser calls are across, retire the vendor and any relay software that existed only to bridge to it.
Build security and compliance controls in from the start
Design these in before the first production call, because retrofitting them means touching every call path twice:
- Authenticated access to the gateway
- Participant permissions
- Handling of sensitive audio
- Redaction of sensitive information
Owning the layer is what lets you answer for requirements such as HIPAA and SOC 2 yourself.
Measure latency, audio quality, and failures throughout
Instrument before you migrate, so you have a baseline from the vendor path.
| Measure | Why |
|---|---|
| Infrastructure latency per hop | Shows whether the new path is faster. Retell went from about 100 ms to about 8 to 10 ms. |
| Audio quality in both directions | Gaps and delays are regressions even when the call connects |
| Call setup success rate | Compare vendor path and gateway path at each stage |
| Calls that fail before reaching your application | The failure your old monitoring could not see. Your SBCs can now count it. |
| Event parity | Each call type emits the same events on both paths |
Common open-source building blocks
Teams that build this layer usually start from existing open-source projects. Widely used ones include Kamailio and OpenSIPS for SIP routing at the edge, rtpengine for relaying media, FreeSWITCH and Asterisk for call control and mixing, and Janus, mediasoup, and LiveKit for WebRTC media. This list is general industry context. Retell has not published which software its gateway uses.
Whatever you choose, the hard part is the ongoing work: staying compatible with many carriers and phone systems, each with its own SIP behavior.
What does running a gateway take after launch?
The migration is a project with an end date. Operating the gateway is permanent. Owning a gateway means owning:
- Telephony interoperability with carriers and customers' phone systems
- Networking and media processing
- Security and monitoring
- On-call operations and incident response
- Capacity planning, maintenance, and safe deployment
Plan the team for year two, not for the migration.
Frequently asked questions
How long does it take to build a voice AI gateway?
It depends on team size, call volume, and how many call features you must preserve. Retell has not published its timeline. The staged approach lets you take value early: routing control and visibility arrive with the SBCs, before the gateway exists.
Do you need a session border controller if you build a gateway?
Yes, for phone calls. The SBC is the secure entry point that receives SIP signaling from carriers and phone systems and routes each call. Browser calls skip it and connect to the gateway over WebRTC.
Why migrate phone calls before browser calls?
Phone calls can be moved by changing routing at your SBCs, one percentage step at a time. Browser calls require your customers to adopt a new SDK, which takes longer and is outside your control.
What is the biggest risk in a gateway migration?
Changing behavior that customers depend on without noticing. Protocol behavior, audio timing, participant permissions, and call lifecycle events all have to match the old path. A staged rollout on a small share of real calls is how you find the differences.
Can you build a gateway and still let customers bring their own carrier?
Yes. The gateway and SBCs sit on your side of the connection. Customers connect their carrier or phone system to your SBCs over a SIP trunk. Retell documents this in its custom telephony guide.
Read the full account
Retell's engineering team describes the vendor problems, each migration phase, and the result in why Retell built its own SIP and WebRTC gateway.
See how much your business could save by switching to AI-powered voice agents.
Your submission has been sent to your email
ROI Result
Total Human Agent Cost
AI Agent Cost
Estimated Savings
Estimate your monthly voice AI cost
Read Other Blogs
Revolutionize your call operation with Retell

Start building smarter conversations today.




