App design (internal, read if you want a headache)

I am building an application to create beautiful interactive graphics. This isn't a game engine; it will be used to create really advanced applications.

In any case, I need to write out some of my design ideas because they are kind of stuck in my head.

Application programming with functional datastructures is different than with imperative or mutable datastructures.

This is because with mutable datastructures, you need to take into account the changes to the basic model such that you can inform the rest of the system about what is going on.

Since functional datastructures are immutable, the rest of the system can use a simple identical? comparison test. If not identical to what it was last time (which is a pointer comparison) then something changed.

Now you can segregate your data into groups, link the disparate pieces through ids and you should end up with a system that has amazing undo capabilities *and* doesn't take a lot of though to program.

Undo is simple and efficient. Just save the old datastructure. Since you know that the datastructures will have large portions of structural sharing, you can bet that your memory usage will grow slowly and efficiently, perhaps more so since with imperative datastructures you have to remember what changed and save this piece of information separately. Redo is similarly easy.

So, basically I intend to have each view simply check the datastructures it cares about and update its internal pieces *if* they are different from what it expects. No events (other than render or update events), no complex transactional-based undo/redo system. Just save the datastructures to named variables and go with that.

It *really* changes the MVC pattern. The controller doesn't need to do nearly as much; it is really much simpler. The views can control how much caching they need by how many references they keep into the main application datastructures. The model can be manipulated very simply and you can still guarantee a lot of details.

Also, using the basic clojure datastructures means I get serialization for free. They serialize pretty naturally.

So enough about that, I need to figure out how I am communicate things to each system.

Looking at the renderer, it simply needs a set of render commands. These commands a very basic, like render object with these shader parameters to this FBO.

Mouse picking is easy, just run the appropriate render commands in selection mode.

The render commands do not need to be generated in the rendering thread; a lot of pre-processing can go on in other threads to generate very specific and fast render commands.

There needs to be some sort of central state that gets updated and somewhat synchronously processed. So a mouse click should generate a request for a hit test. Next time something renders, that request should be processed and the results, should there be any, sent back to the system.

That hit request should translate into some state changes, like an object being outlined or something. These changes can be done in another thread and a new render-command list be generated that does something else.

So, coming up with very specific design guidelines, I guess this is it.

You are going to have a set of N views. When a piece of the model changes, something should analyze this information and update <= N view specific datastructures. These updates can be done in parallel, so there is some parallelization ability there.

These shouldn't happen on the view or render thread, you should do a bunch of pre-processing on other thread such that you build very view specific datastructures that efficiently translate into pixels on the screen. Each view may be able to further break down its updating system but I doubt they could really do this efficiently. It would probably be better to create sub-view-sections and process them independent of each other. Also, they shouldn't update anything in the event that nothing changed that they care about.

The renderer's view specific datastructure is the list of render commands it is rendering.

So lets walk through an entire event sequence.

You have an object on the screen.

mouse down is sent up to the system from the glview.
a hit test is scheduled for the next time something renders and added to the gl todo list.
lets say you hit the object. This is sent back to the system where processing takes place that changes an application datastructure such that an object is selected now that was not.
This selection event is processed and all the views-controllers are told a new model is available. These controllers, running in the systems' CPU-bound thread, do however much pre-processing they can and change some view datastructures. After the changes are done, each control is told to update.

Now you start dragging. This means that the object should move under the mouse (scale,translate,or rotate depending). The view's drag handler should send the drag event to the system where it will do the math required to move the object around. This thread switch may introduce too much latency but we will see. Next the system will update the scene, views that care should update their internal representations (like the render commands). And repeat.

The question is is there too much latency by jumping from the view thread to updating the application's datastructures? Should that happen synchronously while the view updates happen on the separate threads?

Anyway, needed to get some thoughts out of my head.

real hard fbo issue

First off, lets state some the relevant keywords such that a google search has a prayer of finding this...

GL_FRAMEBUFFER_UNSUPPORTED_EXT
glFramebufferTexture2DEXT

OK, so here is the lowdown. It is critically important for most of the really interesting things I want to do to be able to allocate framebuffer objects that render their information to textures.

On the mac, I would create a texture and set it on the framebuffer object and it would work.
On windows, I would always get the above error. Now, pray tell, how would it be possible that textured framebuffers are supported on the mac but not on windows *on the same machine*?

Well, I checked a lot of things. I thought that perhaps this was because one was using pbuffers and the other was using swing's fbo pipleline. I changed the window to be a awt canvas (thus rendering to the native window surface) instead of a swing GLJPanel.

I set up a simple test case using an example from the web (that failed, btw.)

I downloaded lwjgl, setup a test case, and ran an example (that also failed, btw).

I spent hours searching the internet. This is all in my spare time after work, so I get at most 2 hours a day to work on this stuff and that is only a couple times/week. One serious gl problem can set me back several weeks.

I also printed out every single variable I could think of that might affect this operation (pixel store state, pixel transfer state, read/draw buffer status, etc.)

Anyway, to stop holding people in suspense, what was causing the allocation to fail?

It was because I wasn't setting the min and mag filters on the texture before I allocated the FBO.

Min and mag filter specify what the texture lookups should be doing when there are more textels than pixels (minication) and when there are more pixels that textels (magnifying the texture). They are attributes that affect how the texture is *read* from memory, not how it is written.

In any case, in the document for FBO's it explicitly states that mipmapped images are not supported even thought the fbo texture renderbuffer call takes a level.

If you look at the defaults for minfilter and magfilter, they are GL_NEAREST_MIPMAP_LINEAR.

Well, NVIDIA cards are picky about this. Apparently, just setting the min and mag filters makes the texture object a mipmap texture object and the fbo allocate will fail with unsupported (which is true in a pedantic, brittle, poorly thought out sense I guess).

So anyway, a couple evenings of debugging and trying out anything I could think of and the answer is simple. When you want to use a texture on an FBO, set its min and mag filters to either LINEAR, CLAMP, or CLAMP_TO_EDGE. Or probably any constant that doesn't explicitly say mipmap in it...

Jeez what a PITA.