Streaming media files

Ok, i’m just going to give you exactly what i have so far, and what i’m trying to do.

What i’m trying to do, is stream a media file “mp3” from the net. I want to play this file as soon as possible, but also have access to it after it’s done playing.

I’ve read entirely through the opensl es specification manual and pretty much every piece of information i could find about it on the net (including all source code, haha), and even a book which actually got me started with the basics. it’s possible of course that i may have missed something though. your specification manual was actually a million times more useful than i thought would be though (unlike microsofts documentation skills… ;), and without it i would have given up on opensl for sure.

I’m able to stream the file just fine. I can set up a media player with a sink to an output mix and a source as an uri. I pause the media player to start the download, and then use the prefetch interface to find out when there is sufficient data to start playing. This works great, but the problem is after i’m done playing, I don’t have access to the audio. I’m able to play again by either stopping the player and restarting it or using a seek interface to set the player back to the beginning. This lets me know that the audio data is somewhere in memory, and i would LOVE if i could just get a pointer to it (this is what would solve everything for me at the moment). However, i can’t seem to find a pointer to this buffer containing the audio file anywhere.

So, i found a work around. I created another player, now having two (uriPlayer and musicPlayer). I set the uriPlayer to sink to a buffer queue instead. I pause the uriPlayer until the prefetch says theres enough data, which i then create a buffer large enough to hold the data, by calling getduration to get the duration of the streamed media. I use that duration, the sample rate, and number of channels to get the size of the buffer. Then i enqueue the buffer and play the audio player which sends the media (in pcm format, which is the current problem) to the buffer as fast as it can, which was great when i found out that it does this way faster than it does when sending to an output mix. Then i enqueue this same buffer in the musicPlayers buffer queue and start musicPlayer. As long as the player head of the uriPlayer is ahead of the player head of the musicPlayer, everything works perfectly.

I was testing this on a small mp3 file, length of about 16 seconds. I was so so so happy when i finally got it working perfectly. then i tried it on a real song, which was 4 minutes. my program quit when it got to creating the buffer to store the music in. I found it was trying to make the buffer around 180mb, haha. I need to be able to store this audio in an encoded format because pcm is just way way too big, so is there a way to have audio player to play to a buffer in an encoded format? preferably just skipping the decoding process altogether.

Now i’ve thought of a way around this and have started implementing it, but i’m hoping someone here can give me a little insight as to what i should do.

My idea now is that i will add a recorder in between the uriPlayer and musicPlayer. The uriPlayer will play to a small buffer, then the recorder will read from that buffer, encode the audio into mp3 again, and store it at an address or (if using android) a buffer queue, which the musicPlayer will read from. This seems like an awful lot of extra work that could easily be avoided if opensl’s audio player could skip decoding and supply a pointer to memory where the streamed audio is.

I haven’t got this last implementation working yet because i’m sort of having problems getting the recorder to actually encode audio.

The last solution i have, which might actually be the only real solution, is download the file using another api, then supply opensl’s audio player with the downloaded file buffer’s pointer in memory, and just make sure the audio player doesn’t read past where the file is being written. I’m having a big problem with this too because i can’t seem to find ANYTHING about downloading a file using android’s ndk, and the only solutions i’ve come across is downloading the file using java or something. I’d prefer to keep my audio system as portable and contained as possible though, so i don’t want to do that.

sorry for the amount of reading in this post, but i’m happy khronos has a forum i can post on because there really is not much about opensl on the net. Thank you so much for any ideas

p.s. I also made a feedback type of loop, which plays audio as soon as it’s recorded, but there’s a decent size delay (i’m using android right now, and found this information from a site on wordpress or something, which was the second most helpful resource i’ve found, specification document being the first). it’s not possible to have any lower latency, is it?

Hi Iedoc,
What you are trying to do will depend on both the streaming server and the implementation. Some servers have the streaming controls play, pause, etc. at the server, and some allow you to buffer it locally. After the data has been received, it’s up to the implementation to decide where in the media chain it wants to buffer the data.

The reason you don’t have access to the pointer to the raw file data buffer is to protect DRM protected content. The DRM protection may allow you to render the data but not to record it locally in which case you are limited to the buffering of the implementation.

So you tried the first obvious thing, which is to send it to a local buffer queue. And you noticed that PCM data takes quite a bit of storage. I agree that storing the data in encoded format would be best. One option is to create a recorder and record the streaming file to a local file (you could also transcode it this way) and then create a player which plays the data from that file. You need to keep track of the time marker so you don’t hit the end of the file. Keep in mind that this is assuming that you are working with source data that is allowed to be saved.

As far as the latency is concerned, that is mostly dictated by the implementation. You can try playing with the size of your buffers to see if that reduces the latency.

And thank you for the cudos on the documentation!