Bad encoding performance of Linux SBC audio codec

[Christian Hoene]

Hello,

I have made some extensive study on the compression performance of Linux's
SBC audio codec in comparison to the reference implementation.

I am sorry to say that the audio quality of the open source version of SBC
is far worse than the reference implementation.

Can anybody confirm these results?
Do you have any idea why the performance is so worse?

With best regards,

Christian


--------------------------------------------------------
Dr.-Ing. Christian Hoene
Computer Networks and Internet, University of Tübingen
Sand 13, 72076 Tübingen, Germany, Phone [phone]
http://net.informatik.uni-tuebingen.de/~hoene




-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Marcel Holtmann]

Hi Christian,

Post by Christian Hoene
I have made some extensive study on the compression performance of Linux's
SBC audio codec in comparison to the reference implementation.
I am sorry to say that the audio quality of the open source version of SBC
is far worse than the reference implementation.
Can anybody confirm these results?
Do you have any idea why the performance is so worse?

are you talking about the encoder or the decoder. For the encoder, I
can't see the big difference. For the decoder you can clearly hear that
we get the volume wrong.

Regards

Marcel



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Brad Midgley]

Christian

Post by Christian Hoene
I am sorry to say that the audio quality of the open source version of SBC
is far worse than the reference implementation.

If you are using the standalone "sbc" project at sourceforge, then you
should try the version inside bluez. The standalone used to simply be
slower, but we identified & fixed some quality issues too that should
be backported.

[Marcel Holtmann]

Hi Brad,

Post by Brad Midgley

Post by Christian Hoene
I am sorry to say that the audio quality of the open source version of SBC
is far worse than the reference implementation.

If you are using the standalone "sbc" project at sourceforge, then you
should try the version inside bluez. The standalone used to simply be
slower, but we identified & fixed some quality issues too that should
be backported.

the standalone version should not be used at all at the moment. All
efforts should go into the version included in bluez-4.x.tar.gz.

Regards

Marcel



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Christian Hoene]

Hello,

Sorry Bad news.
the encoding and decodng performance of bluez's SBC is really bad.
I just used the latest sbcenc program from bluez-4.17 and compared it to the
reference

Attached you will find wav-files encoded with the Bluetooth reference
implementation and bluez

Filelist
========
compare.sh is the scrip that I use to compare both codec
versions
refsop01.wav the original sample
tmp.ref.sbc with reference encoded
tmp.bluez.sbc with bluez encoded
tmp.ref.sw.wav with reference encoded and decoded (has good quality)
tmp.bluez.sw.wav with bluez encoded and decoded (has bad quality ODG=)
tmp.refbluez.sw.wav with reference encoded and with bluez decoded (has bad
quality)
tmp.bluezref.sw.wav with bluez encoded and with reference decoded (has
medium quality)

Both the bluez encoder and decoder do not work as good as the reference
implementation.

I also conducted some perceptual audio tests with ITU BS.1387 (PEAQ), which
confirm these results.

With best regards,

Christian

PS:
Attached also an enhanced version for sbc/sbcenc.c, which considers more
command line options.

-----Original Message-----
Sent: Monday, October 27, 2008 8:42 PM
To: BlueZ development
Subject: Re: [Bluez-devel] Bad encoding performance of Linux SBC audio
codec
Hi Brad,

Post by Brad Midgley

Post by Christian Hoene
I am sorry to say that the audio quality of the open source version

of SBC

Post by Brad Midgley

Post by Christian Hoene
is far worse than the reference implementation.

If you are using the standalone "sbc" project at sourceforge, then

you

Post by Brad Midgley
should try the version inside bluez. The standalone used to simply be
slower, but we identified & fixed some quality issues too that should
be backported.

the standalone version should not be used at all at the moment. All
efforts should go into the version included in bluez-4.x.tar.gz.
Regards
Marcel
-----------------------------------------------------------------------
--
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bluez-devel mailing list
https://lists.sourceforge.net/lists/listinfo/bluez-devel

[Luiz Augusto von Dentz]

Hello,

Thanks for doing those tests, we never really have a method for
testing the sbc codec, so PEAQ could be a good start. By looking the
results it seems that our decoder is not really in good shape, which
is not a surprise since we normally only use the encoder part. As for
comparison with the reference implementation, bluez sbc codec does not
use floating point as the reference codec seems to be using, so this
may well be impacting on quality, although we probably gain some
speed.

Other points we might need to consider:
- Test with different audio source/parameter. (48000hz/mono/28
bitpool is not that common to be used as reference)
- Mono seems to surfer from quality problems which may cause the problem.
- Consider working on a gstreamer element for doing live tests based
on ITU BS.1387 (PEAQ)
- Also consider using liboil as alternative for using floating point.

BTW, don't assume something is bad just because it produces different
results than the reference, as far I can tell you one cannot really
notice _any_ difference between bluez sbc codec and logitech freepulse
e(enhanced)sbc, so on real world we are not that _bad_.

[Christian Hoene]

Hello Frans,



thank you for the patch!!



Christian





From: Frans de Bont [mailto:***@philips.com]
Sent: Thursday, October 30, 2008 2:34 PM
To: Christian Hoene
Subject: RE: [Bluez-devel] REALLY Bad encoding performance of Linux SBC
audio codec




Hello Christian,

I had a quick look into the bluez sbc decoder test application.
I discovered that the buffering was not correct.
Attached a modified version.

Please note that the output level of the decoder is still 6 dB too low.
This wouldn't be too difficult to fix for the bluez developers.

Best regards,
Frans










"Christian Hoene" <***@uni-tuebingen.de>

2008-10-29 04:30 PM


To

"'BlueZ development'" <bluez-***@lists.sourceforge.net>


cc

Frans de Bont/EHV/APPTECH/***@PHILIPS


Subject

RE: [Bluez-devel] REALLY Bad encoding performance of Linux SBC audio
codec


Classification










Hello,

Sorry Bad news.
the encoding and decodng performance of bluez's SBC is really bad.
I just used the latest sbcenc program from bluez-4.17 and compared it to the
reference

Attached you will find wav-files encoded with the Bluetooth reference
implementation and bluez

Filelist
========
compare.sh is the scrip that I use to compare both
codec
versions
refsop01.wav the original sample
tmp.ref.sbc with reference encoded
tmp.bluez.sbc with bluez encoded
tmp.ref.sw.wav with reference encoded and decoded (has good quality)
tmp.bluez.sw.wav with bluez encoded and decoded (has bad quality ODG=)
tmp.refbluez.sw.wav with reference encoded and with bluez decoded (has bad
quality)
tmp.bluezref.sw.wav with bluez encoded and with reference decoded (has
medium quality)

Both the bluez encoder and decoder do not work as good as the reference
implementation.

I also conducted some perceptual audio tests with ITU BS.1387 (PEAQ), which
confirm these results.

With best regards,

Christian

PS:
Attached also an enhanced version for sbc/sbcenc.c, which considers more
command line options.

-----Original Message-----
Sent: Monday, October 27, 2008 8:42 PM
To: BlueZ development
Subject: Re: [Bluez-devel] Bad encoding performance of Linux SBC audio
codec
Hi Brad,

Post by Brad Midgley

Post by Christian Hoene
I am sorry to say that the audio quality of the open source version

of SBC

Post by Brad Midgley

Post by Christian Hoene
is far worse than the reference implementation.

If you are using the standalone "sbc" project at sourceforge, then

you

Post by Brad Midgley
should try the version inside bluez. The standalone used to simply be
slower, but we identified & fixed some quality issues too that should
be backported.

the standalone version should not be used at all at the moment. All
efforts should go into the version included in bluez-4.x.tar.gz.
Regards
Marcel
-----------------------------------------------------------------------
--
This SF.Net email is sponsored by the Moblin Your Move Developer's
challenge
Build the coolest Linux based applications with Moblin SDK & win great
prizes
Grand prize is a trip for two to an Open Source event anywhere in the
world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Bluez-devel mailing list
https://lists.sourceforge.net/lists/listinfo/bluez-devel

[attachment "sbcenc.c" deleted by Frans de Bont/EHV/APPTECH/PHILIPS]
[attachment "compare.tar.gz" deleted by Frans de Bont/EHV/APPTECH/PHILIPS]

[Christian Hoene]

Hello,

the sound quality got better. Still, both encoder and decoder are not as
good as the reference.


For example, not the audio quality measured with ODB(BV) is

Algorithm
ODG(BV)
In0Filename
In1Filename
PEAQBasic
-1.99
ref.wav
tmp.bluez.wav
PEAQBasic
-1.5
ref.wav
tmp.bluezref.wav
PEAQBasic
-0.82
ref.wav
tmp.refbluez.wav
PEAQBasic
-0.44
ref.wav
tmp.ref.wav


If I use the simple perceptual model of the reference implementation
(option -p), the encoding quality get even better.

Algorithm
ODG(BV)
In0Filename
In1Filename
PEAQBasic
-1.99
ref.wav
tmp.bluez.wav
PEAQBasic
-1.5
ref.wav
tmp.bluezref.wav
PEAQBasic
-0.55
ref.wav
tmp.refbluez.wav
PEAQBasic
-0.17
ref.wav
tmp.ref.wav

Here the old values before Frans's patch

Algorithm
ODG(BV)
In0Filename
In1Filename
PEAQBasic
-1.5
ref.wav
tmp.bluezref.sw.wav
PEAQBasic
-3.89
ref.wav
tmp.bluez.sw.wav
PEAQBasic
-3.88
ref.wav
tmp.refbluez.sw.wav
PEAQBasic
-0.44
ref.wav
tmp.ref.sw.wav

Greetings

Christian



-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Luiz Augusto von Dentz]

Could you please send it as a real patch, I mean in diff format, or
even better if you could configure a git repository so we could pull
from.

Thank you for the contribution.

[Christian Hoene]

Post by Luiz Augusto von Dentz
Could you please send it as a real patch, I mean in diff format, or
even better if you could configure a git repository so we could pull
from.
Thank you for the contribution.

patch is attached.

[Marcel Holtmann]

Hi Christian,

Post by Christian Hoene

Post by Luiz Augusto von Dentz
Could you please send it as a real patch, I mean in diff format, or
even better if you could configure a git repository so we could pull
from.
Thank you for the contribution.

patch is attached.

patch has been applied. Thanks.

Regards

Marcel


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Marcel Holtmann]

Hi Christian,

Post by Christian Hoene
the sound quality got better. Still, both encoder and decoder are not as
good as the reference.

so we do have the sbctester utility in the source code that should be
able to be used for the SBC conformance testing according to the A2DP
qualification.

Regards

Marcel


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Brad Midgley]

Christian

Post by Christian Hoene
the sound quality got better. Still, both encoder and decoder are not as
good as the reference.

It would be ideal to have automated tests for codec quality that don't
require windows and that we could run before commits to track our
progress. We could prepare some encoding/decoding samples with the
reference codec and store them in the project or somewhere online. The
one test we couldn't do this way is to encode with our codec and
decode with the reference.

Are there free tools for checking the quality and for comparing
signal/noise and overall volume? If you can detail the process you use
that would help.

thanks
Brad

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Christian Hoene]

Hello Brad,

Post by Brad Midgley
It would be ideal to have automated tests for codec quality that don't
require windows and that we could run before commits to track our
progress.

Wine will be needed for the reference implementation, which is only
available for Windows.

Post by Brad Midgley
We could prepare some encoding/decoding samples with the
reference codec and store them in the project or somewhere online. The
one test we couldn't do this way is to encode with our codec and
decode with the reference.
Are there free tools for checking the quality and for comparing
signal/noise and overall volume? If you can detail the process you use
that would help.

Currently, I use PEAQ, which is quite expensive (6000 EUR). It provides you
will a couple of parameters regarding the audio quality. I am not aware of a
good open source implementation of PEAQ.

Christian


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Craig Southeren]

Christian Hoene wrote:
..deleted

Post by Brad Midgley
Are there free tools for checking the quality and for comparing
signal/noise and overall volume? If you can detail the process you use
that would help.

The Open Source NetPBM tool pnmpsnr contains some simple code for
calculating the SNR between two graphic images. Doing the same for audio
should be quite simple.

Craig
--
-----------------------------------------------------------------------
Craig Southeren Post Increment – VoIP Consulting and Software
***@postincrement.com.au www.postincrement.com.au

Phone: [phone] ICQ: #86852844
Fax: [phone] MSN: ***@hotmail.com
Mobile: [phone] Jabber: ***@jabber.org

"Science is the poetry of reality." Richard Dawkins

-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/

[Marcel Holtmann]

Hi Christian,

Post by Christian Hoene

Post by Brad Midgley
It would be ideal to have automated tests for codec quality that don't
require windows and that we could run before commits to track our
progress.

Wine will be needed for the reference implementation, which is only
available for Windows.

with the exception that the reference encoder will run into a busy
loop for the first two test samples of the encoding conformance testing.

Post by Christian Hoene

Post by Brad Midgley
We could prepare some encoding/decoding samples with the
reference codec and store them in the project or somewhere online. The
one test we couldn't do this way is to encode with our codec and
decode with the reference.
Are there free tools for checking the quality and for comparing
signal/noise and overall volume? If you can detail the process you use
that would help.

Currently, I use PEAQ, which is quite expensive (6000 EUR). It
provides you
will a couple of parameters regarding the audio quality. I am not aware of a
good open source implementation of PEAQ.

Please look at our sbctester tool. It should do the right thing.

Regards

Marcel


-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/