1 / 34

Embedded System: V 2 IP Phone Design & Development

Embedded System: V 2 IP Phone Design & Development. D96921014 郭人豪. Outline. Introduction Function List Design Obstacles, Workaround and Bug Fix Rule of Thumb Conclusion. Introduction. What is V 2 IP Phone? Voice and Video over IP Phone Extend from H.323

pippa
Download Presentation

Embedded System: V 2 IP Phone Design & Development

An Image/Link below is provided (as is) to download presentation Download Policy: Content on the Website is provided to you AS IS for your information and personal use and may not be sold / licensed / shared on other websites without getting consent from its author. Content is provided to you AS IS for your information and personal use only. Download presentation by click this link. While downloading, if for some reason you are not able to download a presentation, the publisher may have deleted the file from their server. During download, if you can't get a presentation, the file might be deleted by the publisher.

E N D

Presentation Transcript

  1. Embedded System:V2IP PhoneDesign & Development D96921014 郭人豪

  2. Outline • Introduction • Function List • Design • Obstacles, Workaround and Bug Fix • Rule of Thumb • Conclusion

  3. Introduction • What is V2IP Phone? • Voice and Video over IP Phone • Extend from H.323 • Video use H.263, MPEG4 SP, H.264 BP • Audio use G.711, G.726, G.729, AMR-NB, AMR-WB, iLBC, etc. • Large LCD screen with Web Surf AP • 3.5” ~ 7” • Browser, Mail Client, Widget, Media player, etc.

  4. Hardware Design of V2IP Phone • 3com, Agere • Separated Codec & AP Processor • Fast but Complex and High close • TI, Broadcom, Qualcomm • Integrated DSP in AP Processor • Complex Software Designed and Hard to port • Freescale, ADI, Renesas • AP Processor with Multimedia Accelerator • Relatively Low Cost but limited capabilities

  5. Separated Codec & Processor LCD Network I/O Keypad I/OTouchPanel Hook State LED Control Audio I/O AP Processor Modem / Codec CAM Ethernet Switch

  6. Processor Core DSP Internal Bus Integrated DSP in AP Processor LCD CAM Human I/ONetwork I/O Audio I/O

  7. AP Processor w/ Multimedia Accelerator LCD AP Processor ASIC MMAccelerator CAM Human I/ONetwork I/O Audio I/O

  8. V2IP Phone Capabilities • Video • Encoder / Decoder • Multi format support • Adapted Buffer Control • Broken Frame Recovery • PiP, PoP • Lips sync • Bitrate, Frame interval, Framerate, etc.

  9. V2IP Phone Capabilities • Audio • Encoder / Decoder • Acoustic Echo Cancellation • Acoustic Echo Suppression • Jitter Control • Voice Activity Detection • White Noise Generator • 3-way talking, Muxer, etc.

  10. V2IP Phone Capabilities • Dialing Tone • EU, US, JP format • Tone Generator • Human interactive tone generator

  11. V2IP Phone Capabilities • Protocols • SIP protocol (w/ T.120) • RTP, RTCP streaming protocol • RTSP streaming control • SRTP, SRTCP security • HTTP for firmware update & Web-base UI Control • DHCP client, 802.1q VLAN 802.1p, 802.1p QoS • ICMP ping, ARP dup-host detection, etc.

  12. V2IP Phone Capabilities • APPs • Phone Book, Contact List • Calendar • Web Browser • Electric White Board, Screen Cap • XML support for SIP extension

  13. SoftDSP • Idea from Freescale • Using Hardware ASIC for Video Codec • Too complex for embedded system • Core2Duo Process only 3-5 fps for 1080p AVC • Audio Codec is relatively easy • Using AP Processor to encode / decode audio frame • Low cost and can upgrade for upcoming audio codec • Performance is affected by OS and CPU power

  14. Design Requirement • Audio Heartbeat must be called every 10 or 20 ms • Video Heartbeat must be called every 10, 20, or 40 ms • Hook state must be process at once • TouchPanel must be interactive at once but can be executed later • i.e. play click sound at once and insert event into list • While streaming, Audio must be always clear and no glitch • Video is reasonable smooth • Hook state is always first priority • Web surfing or other AP may have performance penalty while streaming is on • LED display pattern is always real-time • And on and on and on…..

  15. System Design Main Phone thread Generate and Monitor Hook thread MultiMediathread ToneGenthread UI Interactivethread SIP thread LED controlthread KeypadRead thread …… Create Events and Insert Event Queue Execute thread

  16. System Design Main Phone Process MultiMediastate Hook state LED controlstate SIP state ToneGenstate Dispatch Idle state KeypadRead state UI Interactivestate Loop Every 10 ms

  17. System Design LED ControlHook state Keypad (Human Interact) Phone Middleware UI Code Virtual Functionsfor implementation I/O Routine Callback functions Network I/O Audio / Video I/O

  18. System Design UI Code KeypadTouch Panel IPC call Phone Middleware Callback functions I/O Routine Network I/O Audio / Video I/O Hook State LED Control Etc.

  19. Comparison of Interrupt / Poll • High Loading • Error resilience or high performance • Low CPU power • How to design queue length? • Mutex, thread overhead, program style • Resource starvation, implementation fault • Timing • How to make sure it runs on-time?

  20. Comparison of Combine / Separated UI • Overhead • Need IPC to communicate • Need CPU to process, waste of already critical CPU load • Complexity • Need to define internal protocol for communicate • OS support, performance • Why choose two-tier design?

  21. Results • Code size • 2.5MB vs. 8MB (plus C++ lib) • Footprint size • 5.5MB vs. 20MB • Client test • 500K bulk dial test passed

  22. Audio Problem • DELAY!!!! • Linux 2.4 use OSS audio driver • 2.6 starting support ALSA audio driver • ALSA provide low/zero latency delay • TIA-810A require phone to have 100ms delay from end to end • Mic → audio driver → audio encoder → packetized → network → muxer → audio decoder → speaker • How to minimize the delay?

  23. Audio Problem • WHERE IS THE CPU POWER??? • VAD use little CPU, so does white noise, most audio codec except G.726. • AES use moderate CPU but still survive • Not in 4-way talking or above • PEOPLE DON’T WANT AES!!! • AEC is much much more complicated • According to a pre-Bell Lab engineer, there are two design in embedded system, 16ms of 1024ms or 64ms of 128ms • The complexity grows exponentially with detect range • How about a phone without Speaker?

  24. Video Problem • Video Input chip not compatible with AP Processor • PAL in okay to transform to CIF • Big problem in NTSC to CIF • 640x240 to 352x288 but no upscale (240 → 288 upscale) • The ghostly Preview Windows • Sometimes, the hardware design is just can’t match the software requirement • Open the preview window 1x1, configure the PrP, close the window, call UI to flush the bitmap

  25. LED Problem • Use AP Processor’s GPIO port and CPU clock to emulate LED on/off period • Not accurate • Actually, totally waste of CPU and out of sync in such a short period • You can’t throw the CPU in the LED routing, there are Audio and Video are waiting for you. • So, how about throw away the LED indicator and use LCD only?

  26. Extension Board Problem • Custom Chip for ext. board signaling and LED flashing • I2C interface, which is buggy and affect the audio chip control • Custom protocol, which is also buggy and need to bug on bug • No-one-can-understand spec, which is told by their senior through phone and personal guide • The packet with no payload after each command • Time interval between commands • Write-read and read-write-read period

  27. LCD Problem • Are you kidding? LCD problem? • Yes, and it’s really a painful experience • CPU spec is for reference only (the pirate code?) • Demo board comes with QVGA panel • CPU spec support up to VGA panel • But not VGA video with PIP!

  28. Amplifier Problem • Not embedded system related but is funny • The gain value write in Audio chip manual is not the actually measured value on board • The auto-gain or pre-gain function on chip has the same name but different implementation with the one in customer requirement • Hardware design and power management

  29. Protocol Problem • The SDP in SIP is a RFC standard • But not in the engineer’s mind of the ATA box designer • Yes, the compatibility problem • But this time, there are no reference, no guide, no document, no source code.

  30. System Problem • In Linux 2.4 you have no clock_gettime() but gettimeofday() • Which is extremely vulnerable while NTP settimeofday() • Embedded system often omit the external RTC clock for cost issue • How can I do? • The invincible watchdog in embedded system • Phone crash, UI crash, power lost, in-experience user manipulation, maintenance, Server crash, etc.

  31. Resource Problem • Flash is used widely in embedded system • Read lightningly fast, write damnly slow • Block all other routine while writing to flash • Buggy interface in Kernel • Multiple write will crash, use fwrite instead • Fwrite can’t measure time • Configuration lost, Bad bits in flash, etc.

  32. Rule of Thumb • Customer don’t know what they want, just push the product and ads to them! • Likewise in sales. Make sure you do not admit every function what they just think of while stay in toilet. • Embedded system is customized and extendable • But they are NOT everything. Think before agree the requirement from customer • Don’t argue with engineers • Try to help them workaround the bug or negotiate with customer to have another way

  33. Conclusion • Choosing RIGHT BSP will shorter the development quite a lot • Think about what you’re designing before you design • There are always a workaround there, just you may not find it • We are developing embedded system! Throw away the compatibility! • Choosing the RIGHT HARDWARE is the same important as good code-syntax writing • Boys, this is not the well-known x86 system • Please always keep in mind that the CPU performance is limited while coding on embedded system • The RAM is limited too, don’t memcpy() on every stuff!

  34. Thank you • Please have some feedback.

More Related