Streaming speed tests

Jan 6, 2015 at 5:46 PM
Hi,

I'm developing a media player for an existing system, where the client is a wpf application and the server is running on a windows service (self hosted). In my scenario, the data is recorded by this existing system, my server opens this data and streams it to the client when requested.

There are two distinct ways the data is served: when the client requests recordings for playback or for exporting data. The difference is that, when the client wants to export data, the server will then compute the data and knows the size of the exported files it needs to send. In the playback scenario, there's no way to know the size of the files - in fact, there are no files per se, just an open stream where the server writes the data for the client to read.

After some research, I implemented the playback scenario using push stream content. This was the only way I could get the streaming to work over a single request / connection.

When the time came to implement the export scenario, I realized that the download speed was very low (using push stream content). Since the server pre-processes the data and packages it in a neat .mp4 file, there is no streaming involved. It just needs to make this file available for the client to download. In this case, this exported file won't be reproduced by the client, but merely saved locally, so the client has to download it as fast as it can - this is where I realized the speed was horrible.

As a palliative solution, I implemented the exporting of these files via static files on the server. This method yielded speeds 20x faster than the push stream content method (in the network I used to test, the speeds were around 500 KBytes/sec and 10 MBytes/sec). While running tests, I also realized that the stream content method was also very, very slow.

With the export download speed as high as I could manage, and running on an acceptable level, I began researching on the push stream content method to get it to run faster, since my playback scenario requires that I use it. After some tinkering, I managed to get comparable speeds, close to the the static method performance. What I did was use a big buffer to hold data and write it less frequently and with heavier loads into the stream.

I wrote a simple server / client project that demonstrates a stripped down version of what I'm actually working on, it's available @ StreamingSpeedTest. It demonstrates how I'm streaming data, and compares the different methods' (push stream content, static files and stream content) download speeds. To get a better view of what's happening, it's better to run the server and client on different machines.

My questions are: Is there a better way to stream data? Are there any other techniques I can implement to improve on the speeds I'm getting? And why on earth is the 'simple' stream content so slow? I just seem to be missing some information and / or I'm making some mistakes on these implementations.

Sorry for the very long text. Any help or comments are welcome, thank you!
Jan 11, 2015 at 11:06 AM
Took a brief look at your code...

Firstly, if you want to test the speed of the server on it's own, don't read your data from a file but instead generate the bytes. This will give you a better indication as to the performance of the server without possibility of disk i/o as a bottle neck.

Secondly, looking at steamer.cs, I suspect your implementation is sub-optimal. While your stream is async it is lacking parallelism - while you are reading the file to the (large?) buffer, you are not writing to the stream. And when you are writing to the stream you are not reading the file in the buffer. Addressing that, I feel, would yield better performance characteristics (and is trickier to write).
Jan 13, 2015 at 10:00 AM
Hi damianh,

Thank you for taking the time to help me out. In response to your comments: the reason I'm reading the file is because that approximates better to my real world scenario. My idea wasn't to get super accurate download speed values, but to compare the download speeds using different strategies. About the streamer class, I get what you're saying. Again, I left it like that because it simulates better my real scenario, which is more complex. I felt like that stripped down version of the streamer class does a good job to help me exemplify the behavior I was experiencing with my real system. If you take a look at the code, in the class Common.Utils.Extensions, you'll see I (tried to) apply this technique by using two buffers and alternating between them. This particular code is used to copy data between streams.

The whole deal I wanted to point out is the great download speed differences between push stream content, stream content and server (in my case, the subset static) files - it just feels like I'm doing something very wrong here, and thus, my questions in the first post.

ps.: by 'large buffer', I'm refering to Common.Utils.Constants / reference

Thank you!
Jan 30, 2015 at 10:25 AM
Sorry to bump my own thread, but has anyone cared to take a look at this?