Help needed getting MP3 OutputMixer working on A1S AudioKit

[level0dev]

I'm trying to play two MP3 at the same time using an OutputMixer, but no matter what, my playback is too fast AND I get messages of insufficient buffer.

I tried increasing the buffer in minxer.begin() but while the speed might be changing slightly (it's hard to know for sure, the playback is still way to fast) I keep getting buffer increase messages in the serial monitor.

I tried running the mixer with only a single file (OutputMixer<int16_t> mixer(i2s, 1); and all mentions of audio [2] commented out) - it plays slower than when I try to play two files, but it is still too fast.

I've seen some mentions of buffered stream in the discussions, but not sure how I could add it here (and if that would solve the issue).

Thanks in advance for any and all help

Current code and mp3 files I use below:
Static_master.mp3
test2.mp3

#include <SPI.h>
#include <SD.h>
#include "AudioTools.h"
#include "AudioTools/AudioLibs/AudioBoardStream.h"
#include "AudioTools/AudioCodecs/CodecMP3Helix.h"

const int chipSelect=PIN_AUDIO_KIT_SD_CARD_CS;
AudioBoardStream i2s(AudioKitEs8388V1); // final output of decoded stream
OutputMixer<int16_t> mixer(i2s, 2);

//audio [1]
VolumeStream volume1(mixer);
EncodedAudioStream decoder1(&volume1, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier1; 
File audioFile1;

//audio [2]
VolumeStream volume2(mixer);
EncodedAudioStream decoder2(&volume2, new MP3DecoderHelix()); // Decoding stream
StreamCopy copier2; 
File audioFile2;


void setup() {
   Serial.begin(115200);
   AudioToolsLogger.begin(Serial, AudioToolsLogLevel::Info); 
   

  //setup i2s / audiokit before SD!
  auto config = i2s.defaultConfig(TX_MODE);
  config.sd_active = true;
  i2s.begin(config);

  // setup SD
  SD.begin(chipSelect);

  //load files
  audioFile1 = SD.open("/demo.mp3");
  audioFile2 = SD.open("/Static_master.mp3");

  // set initial volume
  volume1.setVolume(0.8);
  volume1.begin();

  volume2.setVolume(0.5);
  volume2.begin();

  //setup decoders
  decoder1.begin();
  decoder2.begin();

  //start mixer
  mixer.begin(2048);

  //
  copier1.begin(decoder1,audioFile1);
  copier2.begin(decoder2,audioFile2);


}

void loop() {
    
    //copier loop
    copier2.copy();
    copier1.copy();

}

[level0dev]

Well, I can't seem to make two mp3s to play correctly with either input or output mixers. Tried buffering, tried resampling - the output is either sped-us mess or full of gaps and only playing one mp3.

I know that I'm far from a good programmer and chances are I'm messing something up in code, but at this point I think ESP32 can't really mix mp3s. I mean, at least one of the methods I tried by modifying the existing examples should have gotten me somewhere closer.

[pschatzmann]

Mixing with a mp3 file is challenging so mixing 2 mp3 files is even more challenging!

Not sure what the best approach would be: Maybe it is a combinations of splitting the decoding result up for one file and using a separate task to keep a decoded queue filled and use that one as source for mixing. Or use 2 filling tasks and one mixing task.

[level0dev]

Thanks Phil. I should have expected it being much more complex than I thought after a week of not finding a single example of ESP32 mixing mp3s.

I appropriate your recommendations. However, I don't feel skilled enough to tackle something that advanced at this point. I shouldn't be trying to implement multitasking when I'm still struggling with syntax! 😆 Maybe I'll come back to it one day when I get more familiar with writing code for ESP32. For now, I will investigate some alternative approaches (mixing mp3 with wav or stick with the generated audio to tackle the radio static, but make it more interesting than just noise).

So, at this point I'm marking this as answered with the answer being: mixing two mp3s on ESP32 is not doable at this point

However, I welcome anyone who wants to prove me wrong!

[pschatzmann]

Or you could replace the mp3 with simple wav files to make things much simpler

[level0dev]

It won't work for what I want it to do with wav only - I want bunch of old time radio shows stored on the sd card, and even at the quality they are, they would require too much space in wav format.

I can use wav for the static, as it's going to be only a couple of files, so I'm hoping that mixing mp3 and wav could work.

[pschatzmann]

For that purpose I think you can even reduce the sample rate down to 8000 mono w/o noticing any quality loss...

As described above and in other discussions, I recommend to run the decoding of mp3 to a MemoryStream to split the decoded pcm data up into smaller pieces and do the mixing in a CallbackStream.

[level0dev]

True, and they already come in mono (and usually 32kbps and 22.050 kHz), so it's not like they are high quality to begin with (the "masters" we had is tapes from 30s and 40s).

However, a quick conversion to wav makes the file 11x larger:

Considering that I would end up having hundreds of files I don't think wav is really feasible, even if I take the bitrate down even more.

I'll read up on the MemoryStream and CallbackStream, but I'll look into alternatives I mentioned above first - I really need to get a better hang of writing C and working with ESP32 and this library before trying more advanced stuff like this, especially after feeling a bit down after hitting a wall on the first step of my project ;)