chrishowie.com

Not creative enough to think of a better title right now…

I just spent almost two hours hacking away at the Banshee visualization pipeline again and made one very important optimization: when the callback function is null (which happens when nobody in managed land is listening for visualization data) the visualization pipeline is effectively disabled. The only element that remains active is a queue, which provides a 5-second buffer so that visualization can be quickly resumed.

The patch is now over at GNOME’s Bugzilla, just waiting. Waiting for Aaron to commit it. Waiting for a chance to prove it’s all grown up now. Hoping to bring joy to audio enthusiasts all around the globe.

Yeah, I did watch Tommy Boy recently. … Why?

So apparently the visualization stuff in Banshee has been disabled since it’s a CPU hog. I don’t know why I didn’t notice it during testing (and I did check) but there seems to be a five-fold increase in CPU utilization with the visualization pipeline running. For me, this is an increase from 1-3% to 10-12% on dual-core 2.6GHz amd64.

After ruling out a few things I discovered the root cause. In OpenVP, PCM and spectrum data are represented as 32-bit floats, which means that the pipeline needs to convert whatever the audio format is into this one. Along the way it resamples the stream too, to provide a consistent frame rate of consistent-size slices. 512 samples 60 times per second is 30720 samples/sec. If anyone can show me a song found in the wild encoded at this rate I’ll give you… well, nothing, since I’m a college student and couldn’t afford to give you this pencil I have sitting on my desk.

Anywho, the conversion to float cannot be avoided but the resampling can be. By setting it to the more common 44100Hz sample rate CPU usage dropped to 4-6%. It’s pretty cheap to convert formats and throw duplicate data around, but interpolating data is a lot more expensive.

At some point the vis pipeline will be smart enough to split music into chunks of a size depending on the current sample rate. Until then this patch should be good enough. (I hope.)

After a few hours of hacking using a wigdet that Michael and a few others pointed me at, I now have a working Banshee.OpenVP extension!

Much work still remains to be done, but what I have is a pretty stable foundation. Both the OpenVP 0.0.1 branch and Banshee.OpenVP are available over at the Google Code repo. They should both be stable enough to test, but since the build system isn’t in place yet I’m not asking the general public to try it out yet. But if you know your way around MD, autotools, and Banshee you might be able to get it working. (Please don’t ask for help with that part yet. If you get it working and hit a bug please let me know though.)

Banshee.OpenVP is on hold indefinitely until a stable GTK+ widget providing an OpenGL context is available. All of the wrappers I have tried have serious issues that prevent this project from even making minimal headway.

I’ve been working with several Banshee people this week and we finally got my patch committed. All this patch does is provide a managed event that acts as a PCM and spectrum data source. While simple, it is a required building block for the upcoming OpenVP extension.

Right now I am getting ready for the 0.0.1 release of OpenVP. The version number is intentionally scary; the API is far from stable and will be changing rapidly. The purpose of this release is to provide a stable target for the Banshee extension. Hopefully getting this in Banshee will generate interest in OpenVP, as well as giving Banshee some cool eye-candy. Everybody wins!

My understanding is that after OpenVP 0.0.1 is released and the Banshee.OpenVP extension is finished (it’s proof-of-concept quality right now) it will be absorbed into the Banshee source tree and maintained over there. Which means it will likely come with Banshee 1.4 by default. Awesome.

(Oh, and OpenVP has migrated from my personal Subversion repository over to its own Google Code project.)

OpenVP, Banshee.

A week or two ago I was in #banshee chatting and posted a link to an album of some OpenVP screenshots. The developers were interested and asked me if I would try embedding OpenVP in Banshee. I have since submitted a patch that adds an interface that PlayerEngine classes can implement to become a visualization source, and also implements this interface on the GStreamer backend.

Once OpenVP is library-ized there will be a second patch adding an OpenVP extension, which will make Banshee do something like you see in the above screenshot. Probably not exactly as you see it there, since I’ll probably want to “take over” the Now Playing window and render there instead.

Basically what I’m saying is that all the ugly plumbing is done and right now it works. It just needs polish, and hopefully it’ll be ready when Banshee 1.0 is.

While thinking about how I was going to implement something in Kaffeeklatsch recently, I took a step back and looked at the class structure. What a mess. Over the next few days I drafted a new class structure that would better reflect the goals of the project and make implementation of new features simpler.

The last week or two have been spent implementing that new design and it’s very promising so far. Specifically, media player connectivity and presets have been overhauled. The OptionAttribute class has been removed in favor of a series of attributes from System.ComponentModel. The GUI designer is looking more slick than ever.

With the rewrite comes a new name: Open Visualization Platform, or OpenVP for short. Development is taking place in my public Subversion repository, so if you want to track it just svn co https://layla.chrishowie.com/svn/OpenVP. You will need a recent MonoDevelop to build the project.

Over two nights of hacking I implemented a movement filter that’s compatible with the AVS Dynamic Movement effect.

Google Video was giving me grief, so I posted it on YouTube this time:

Lately I’ve taken to hacking Affe quite a bit. Just a quick list of a few changes:

• Value types can now be the target of invocations.
• A value type will be automatically boxed when being cast to an interface that it implements.
• Any object type can now be unboxed. This fixes the case where a value type boxed into an interface type could not be unboxed without being cast to System.Object first.
• Any expression can be used as a boolean. Reference types will be compared to null, and numeric values will be compared to 0.
• Added the &&, ||, and ?: operators.
• null is now a keyword.
• Fixed a parser bug that would cause syntax errors when the sequence “(identifier)” was encountered and did not begin a CastExpression.
• The == and != operators can now be applied to reference types.
• OperatorExpression now checks for operator overloads and uses them when present.
• Arrays and types with indexers can now be indexed.
• Added support for single-line (//) comments.
• Added a “local bag” which is used when building locals.

The last item could use some explanation. It’s a size optimization really, and probably unneeded, but I’ve always thought something like this would be neat for mcs to have. This implementation is my proof-of-concept.

There are a lot of things that require extra locals to be defined. For example, if some method returns a value type and you’re directly invoking on it (for example, somestring.Length.ToString()) then there are two IL sequences that could be used. Assuming the int is on the top of the stack, the first is box System.Int32; callvirt instance string object::ToString();. This is what seems intuitive, but there’s a reason mcs does not do this. Boxing creates a new object, and that object must now be garbage collected. It’s much more efficient to use the stack for value types. So instead of boxing, a new local is declared, say “temp”, and this IL is used: stloc temp; ldloca temp; call instance string int32::ToString();. No boxed object, no virtual method resolution, no additional strain on the GC, no wasted heap. But now we have another local.

What if you hard-coded ten such calls? That’s right, you get ten locals. The JIT may be smart enough to optimize them out, but the IL is still going to be bloated. That’s where my local bag comes in. When such an expression is compiled in Affe, it asks the compiler state object for a local from the bag, for temporary use. The state will look through a list of unused locals it has and will return the first it finds of the same type, but will keep it in the unused list (since the callee said it’s temporary). If none can be found then it will create one and add it to the list. (If the local was requested for permanent use then it would be removed from the list if it was there, or if it wasn’t then it won’t be added to the list.)

The upshot of this is that if you make 10 hardcoded invocations against a value type that’s being returned from somewhere (or even a constant) they will all use the same local, assuming they’re all the same value type.

Now consider the case where you have this code in C#: if (foo) { object o; ... } else { object o2; ... }. This is admittedly contrived, but it does the job. We have two scopes with a variable of the same type. With mcs you get two locals. With Affe you get one.

To maintain scoped variables, the Affe compiler state maintains a stack of symbol tables. Each block has its own table that is pushed prior to analysis, then popped, and again for the compile pass. When a table is pushed, the state will check for symbols that correspond to locals and will remove them from the local bag if present. Then when popped, all the locals are added to the bag. So in the example above, the “if” block’s table is pushed during analysis and a new local symbol is added. Then it’s popped, and the local gets put in the bag. When the “else” block’s analysis is run and it asks for a local of type object, the one from the bag is handed to it.

It’s not a terribly major optimization, and I expect in Affe’s case it actually takes more time during compilation than it saves during runtime, but I’m interested if this would be interesting for the mcs hackers. I imagine it would eliminate a fair amount of IL bloat.