Tuesday, April 22, 2008

Been a long time

I bit off more than I could easily chew at work recently.

Actually, I have been really working hard for about 3 weeks, with last week being the culmination and the hardest, and this week I am ramping down.

We haven't refactored the code base for so long, and I decided to do something to address a few outstanding technical issues with the product that ended up being a lot more work than I had originally accounted for.

This is the first time I have worked with a multiple document system. Add to that the fact that we keep the files in a datastructure that matches their structure in memory. There is no translation step on load or save. This means that links (urls) between files are just strings, they don't point at anything. If you change a document's location, or you move a resource, you had to run through the entire project looking for links to the renamed document or the moved resource to update them.

Needless to say, this didn't scale well.

Now, we parse urls into special objects that build a graph of the documents, the objects in the documents, and how they link to each other. We build a document graph. This sort of low level change, in a product as sophisticated as Blackfish, really takes a lot of work. It reduced complexity of move, rename, and delete operations considerably; but it increases the static complexity of the system. Anyway, it was hard, but it works and it is cool, clean, and fast.

I also had a bit of a breakdown physically two weeks ago. Sometimes I will train various different sports until I enter a state of physical exhaustion.

Real exhaustion isn't something that happens after you run a long time. You may be temporarily exhausted, but it takes weeks of overtraining to reach a true physical exhausted state. There are several key indicators you are overdoing it, but I don't usually notice them until they are pretty far along. High heart rate, even during the evening. Failure to sleep effectively, moodiness, pretty much all conditions I have without being exceptionally drained. You also stop healing effectively which is the largest problem.

Anyway, I pretty much took 2 weeks off from sports. Getting a very time consuming video game (Space Rangers 2) helped a lot. This game is pretty much all about zoning out wandering around the universe checking shizzle out. There is a point to it, but you can achieve it many different ways and there isn't a use to rushing. That and I am dancing with a very sweet girl from salsa so we just danced a lot (although I find it really tough to detach from hard work enough to dance well).

And, somewhat passive aggressively, I missed a meeting yesterday and just went to a baseball game. This pissed people off, but I needed the rest so what can I say? I got a somewhat direct email from the boss today telling me to either show up for the meeting or reschedule it. I don't intend to respond.

Anyway, I have started the physical stuff again, and what I am working on is wrapping up. The next 7 weeks should be OK for me. What I really need is some serious rest from this product, but I don't see that happening any time soon.

So, I am officially employed by NVIDIA. A large pay raise, and employee badge, and meeting some pretty cool people are what I have to show for it. Now I need to pay off the first loan on my house (the small one that ensures I don't have to pay loan insurance) and refinance the house. Rent it out and buy another.

Chris

Tuesday, April 1, 2008

F#

This language is a whole other world.

I bought a book on it (expert F#), and I will shortly buy another book on it (Foundations of F#).

After LISP, I didn't know if I could ever really be with another language, really. You see, I saw so much promise and possibilities in LISP.

The look of the code was amazing. The smooth and perfectly elegant lines of code, mixed with the intricate and somewhat arcane lines of meta-code. It still gives me chills to just think about it.

The infinite possibilities of a programmable programming language. Who wouldn't fall in love with LISP? Surely, I cannot blame myself for such fantasy.

Over time, our relationship soured. I yanked and urged common lisp to move into somewhat common territory (with a name like that, you would think it would be easy...). I kept thinking that I could somehow change it, to make the infinite possibilities I saw with it really happen.

The final straw was moving it to the .net platform. This is a necessary move, it would have been good for both of us. The next step.

There comes a point in every relationship where you must either move forward or move away. And as of late, I must admit that my thoughts have been absent and my intentions toward another have grown stronger.

I didn't even know who that other was.

Then one glorious day when I felt the need to express some of the joy that life had brought me, I caught the look of something I had seen from afar but never considered.

F# and I are curiously staring at each other over coffee at the moment. It appears that we really don't speak the same language; although when push comes to gentle, rhythmic shoving I am sure we will figure it out. We are so very viscerally attracted to each other that maybe we just have to get over our differences one way or another.

This will be a sweet and gentle love affair at first. I need to give myself time to adjust to new concepts, to the gloriously dirty feel of doing something very familiar with a very unfamiliar body.

Perhaps F# can help teach my wounded heart to love again...

Chris

Saturday, March 29, 2008

That was fucking sweet

Had a blast last evening.

Too hung over to talk much about it.

Managed to get two awesome people to hook up with each other.

Have to work today and I am way too tired to do it so I will need to nap, but I am satisfied with the day already.

Chris

Thursday, March 27, 2008

Busy Life

I am right now feeling pretty drained from a long, intense conversation with Ceci. We met at a coffee shop, we went over problems we were having on a professional level, and we thought about ways to address them. For 3 hours, we spoke at very deep levels and it was very healthy and gave me a lot of things to think about, but it also really took my energy away.

I was hoping to go to the Avalon with my dance partner, but she is busy dealing with a persistent ex-boyfriend. She is dating someone who sounds wonderful, so it isn't a situation where the ex has a chance but he isn't giving up easily (which I find admirable). She will have to put him down hard, and I think tonight it will happen. But I needed to just forget about the stresses in my life and dance, so in a selfish way I am totally bummed.

So because I am overwhelmed and tired, this post is going to be more self-centered than most of my posts, which is saying a lot.

This is my current schedule:
Dance three times a week.
Jiu Jitsu two to three times a week.
Soccer twice a week.
Run once or twice a week.
Personal workout (pushups and back exercises) once a week.
Pushing a product to release in the next few months (roughly 40-50 hours/week commitment).
Serious practice singing, 1 hour or more once a week-ish.

I don't have time to breathe. The problem is, each of these things are things that when I am done I always think "Man, I really love this".

There was a time in my life where I was playing a lot of video games but really not achieving anything else. At one point I made a commitment to stop playing video games and other distracting things and just work towards being places in life where I really want to be.

But how much can a person really absorb in a daily basis? What is a limit on focus and effectiveness? I know that I can't do very good computer science more than 6-8 hours in a day. I just loose the ability to focus very hard after a while. I have about 4 hours in the morning where I am extremely focused, and probably the other 2 to four are give and take.

But I don't find that doing other things significantly detracts from my work. I find that the combination of all these things allows me to handle situations I otherwise would have difficulty because I have so many other things I can focus on, and at least one of them is always a nice distraction from something.

You could argue that my life is indeed filled with distractions, and this isn't allowing me to deal with personal issues that I really need to deal with. But the simple fact is that I am someone who really likes to get things done. When I stop and smell the roses, as people like to put it, I sneeze and get bored. So in some ways, I don't give a shit that I am not dealing with personal issues. I have them, everyone does, and so be it.

I try to be open to whatever opportunities life gives me, and just go with the flow. The problem is that the opportunities you are given aren't always the opportunities you really want. The question is are they the opportunities you need?

I would like, at some level, to be in a relationship (if my previous posts didn't give you and idea how painfully badly I would like this). But I have had so many disappointments and upsetting situations regarding relationships that I really don't know how much effort I am willing to put into this.

It feels like I want that opportunity, but life isn't cooperating. So I am not complaining, I am just going with what is given.

I have noticed that in general you can't capitalize off an opportunity you aren't ready for. So sure, a percentage of success is luck. But there is a hell of a lot of preparation involved in being ready for that opportunity and being able to make something of it when it happens.

Muhammad Ali used to say: "The fight is won or lost far away from witnesses - behind the lines, in the gym, and out there on the road, long before I dance under those lights. ". I really like that quote as I believe it signifies a large portion of my life.

The question becomes how much effort have I put myself into w/r/t accepting a relationship? Is it worthwhile to assume that my lack of success in this area is a result of not doing the groundwork required to be able to capitalize on these situations when they occur? Perhaps it is in not being in a position to even get the opportunity in the first place? What exactly does putting yourself into a position of being ready for a relationship really mean, exactly? Does it mean working on empathetic communication styles? Does it mean ensuring you will be able be a resource and not a burden on the person you are with?

Anyway, that is just an observation. When it comes down to the details, I am not sure I want someone sleeping next to me at night. I enjoy sex, don't get me wrong, but I don't always enjoy having someone else in the bed at night. Perhaps I just think I want a relationship because that seems like a logical thing that will make me a little less lonely. The grass is always greener on the other side of the fence, however, and usually when I am in one I am not measurably more happen than when I am not. Only at the beginning, when I am still infatuated with the person am I really pumped.

How much of my loneliness comes from an inability to connect with lots of people? Perhaps I am attempting to mask my inability to connect with a lot of people by having one person I have a super connection with. Is this really the healthiest thing I could be worrying about, or could I perhaps worry about connecting with people in the general sense?

What exactly is the goal of all this? What does being healthy really mean for any one individual? Does it mean you are happy? Does it mean you are content?

I have noticed that I grow the most in life in general when I am the most unhappy. There is some significant part of adapting to new situation and learning things about yourself that involves just fundamentally being uncomfortable, if not unhappy. I would say that my strongest trait is my clear vision, but a close second is my ability to adapt and try out odd situations. But then, to fully utilize this second trait don't I need to continually put myself into tough situation where I am not going to be the happiest person in the world but I am definitely going to learn things and grow to be a greater person than I was?

I leave with you an observation, which seems appropriate considering the number of questions I listed. Rarely are the people who contribute most to society by my measures the happiest (or healthiest). So at some level, perhaps you need to weigh your ability to be ok with tough situations with your desire to contribute great things.

Chris

Monday, March 17, 2008

Graph-based Programming

So I was really out of it tonight. I did yoga today for the first time in a very long time in a super hot room. It felt amazing, but I was pretty zoned the rest of the day.

Anyway, I am trying to decide how to spend my time.

I love LISP, and more the idea of super sophisticated metaprogramming systems. I spent several hours yesterday and today looking over the iron python code, and I had a very interesting thought.

Python is by no means a complex language these days. Yet the python .net compiler is a seriously tough project to get into. It may be faster than CPython, but it is really tricky trying to figure out how things work. The craziest thing is how much random code there is in the compiler doing various things, a lot of it very very repetitive. Wouldn't a simpler language with more powerful metaprogramming abilities be capable of overwhelming a complex language with less sophisticated metaprogramming abilities?

Lets say you have a sophisticated problem X. The closer your language looks to X, the less work it is going to take to solve it. Couldn't a compiler be seen as a general solution to all problems that then has to be customized to the particular problem? Now you could add all the crazy complexity to the compiler to make X very easy to solve. Then your solutions probably would not generalize to making problem Y very easy to solve. This is fundamentally why a given language has a given advantage in a given problem domain.

But a programmable language should be capable of being used efficiently for any problem, because in a sense, you are building your own compiler just for that problem.

The point is that I could see removing 90% of the structure and complexity of Iron Python, replacing it with general high-level metaprogramming capabilities, and ending up with a simpler system. A great test would be to write an Iron Python compiler in super-meta-language Y and see how much shorter it is.

Anyway, implementing some sort of LISP on .net is something that I am seriously thinking about.

Next on the list of really interesting things to fuck with is writing a graph-based graphics manipulation program and writing a book about it. I know a ton about large scale editing applications that very few other people know. I don't know how generalizable this knowledge is to other fields, or how useful it would appear to other people.

But I have to believe that speaking clearly about the technical advantages of the entire system we have designed for editing applications should elicit quite a response in people who like 3d graphics, who do like graphs, or who just like to think about things that blow your mind a bit. Plus it is something I can write about with a high degree of confidence that what I am saying is at least feasible. And I didn't have to pay for the feasibility test myself (thanks Anark!).


So, I was really sitting there completely in my head in a dark room thinking about all the advantages of a graph based editing systems, and I started thinking about representing programs a graphs, and could I build a lisp compiler that had macros, but that also let you run code generically operating on the representing of the code as a graph before it is dumped to IL (CLR assembly code)?

What does a program look like in its graph state? It should look something like a callgraph. Along with this (or embedded in it), would be a graph of what datatypes are used in which functions (or perhaps just allocated). And the datatype graph (schema) described by the program. Also, there is the syntax tree which hooks into the callgraph.

Now merge this with the datatype graph created by reading an xml schema. You could have extend the pure language directly with the datatype graph you are working with. This would be a code generator of sorts; but it would be a much more general one because your base language is already completely graph based. Talk about the results fitting into the language. You could build systems like this that were technically indistinguishable other programs but still extended the programming language in absolutely crazy ways.

Anyway, it is interesting to think about programs and think about graphs made up of every possible piece of related information. It is also interesting to think about the possibilities of being able to, at compile time, to manipulate those graphs to a great degree. Why not run time?

Because I really don't know how far I could push .net during run time and still have things work. Can I dynamically generate classes? Where does the code go? Sometimes you need a dll to be saved to disk in order for things to work, I know that, but the Iron Python guys may have solved a lot of those issues. Plus, I haven't really figured out what it would mean if you had access to all this stuff, and compile time just feels a little more natural to me.

Why have we designed all this structure around systems that are fundamentally graph based to begin with? I am not saying that visually connecting lines is a good idea, I never really liked graphical programming techniques. What if you built a runtime system that was fundamentally made to run these graphs, just as the current systems are made to run assembly? Then you would have all the crazy expressivity of the graphs all the time. It could do all sorts of optimizations based on which edges were traversed more than other edges, which ones were never traversed, etc. It could really morph itself both to new demands and new problems. I guess I should take a compiler theory class and find out just how far you can really take this idea.

Chris

Saturday, March 8, 2008

Axiomizing .net

I have been thinking about what lisp would mean in terms of implementing it on a .net framework. Here are some conclusions:

The DLR (dynamic language runtime) provides four basic services (http://en.wikipedia.org/wiki/Dynamic_Language_Runtime):
1. Dynamic type system.
2. Dynamic method dispatch.
3. Dynamic code generation.
4. Hosting API.

The power of lisp, above and beyond other languages (for me), comes from a only three things:

1. Syntactical Simplicity. Lists of lists. This enables levels of metaprogramming that would be impossible with a complex language.
2. Macros. The ability to extend the language in ways that do not look different than any other lisp code.
3. Minimal language specification. Given 6 keywords and macros, you can build all of common lisp. It really has axiomized computing down to the basic primitives required. Everything else is gravy.

I think an appropriate LISP implementation using the DLR would be the most minimal LISP API that exposes a self hosting compiler/environment (thus macros work) as well as access to all 4 DLR services.

I think it would be awesome if you had the most bare-minimum wrapper that included just enough to work and very raw access to everything else. Then people could build macros that extended the compiler with better support for .net paradigms.

I guess what I am trying to say is that I would write a system that built, layer upon layer, upon the lowest possible denominators. The fact that you can extend the compiler easily means that I, as the language implementor, should implement bare minimum full API access to the four services. I assume that everything else comes through those services, and the more I can expose them the better.

This would be a LISP for experts. In order to use the rawest version, you will need to have an excellent understanding of the CLR and DLR. It would make literally anything possible, however.

That is the theory, anyway. Implement the bare minimum that makes supporting the full .net system theoretically possible, and then thoroughly document the implementation and its implications. This LISP wouldn't be a common lisp implementation on .net. I would have lists, car, and cdr, but not much else. There is no need for hashes or vectors or anything like that, .net already has those.

It comes down to how do you do the least amount of work in order to gain all the important advantages of LISP along with the important advantages of the .net framework, the CLR and DLR. Absolutely no 'niceness' wrapping at the lowest level, just bare minimum axiomatic access to the base services.

Perhaps using the core system, writing a standard .net class takes 5 times the LOC that it does in C# because you have to explicitly specify all the assumptions the C# compiler uses. The key point is that if you have macros, it doesn't matter at all because clients can just write macros and such that make the same assumptions but the underlying language allows experimentation with very different assumptions.

Anyway, I don't know how closely the current .net LISP matches this paradigm. I will have to study iron [python|ruby] a bit to gain a good idea on what a good .net language implementation should look like, and then go over the existing LISP one(s).

Think of the power of axiomizing support for .net. It should be similar to the power of axiomizing computing just translated into a different domain. Perhaps there are crazy ways of defining properties that csharp doesn't allow. Perhaps there are class extensions and clever things you can do that you can't in any language but that aren't explicitly forbidded in the .net system; you just don't have low level enough access to use them.

The point is that because LISP enables endlessly powerful metaprogramming, language implementors should never make any assumptions whatsoever. They should write to the exact .net platform specification, and let macros upon macros simplify the low level description into useful abstractions.

Chris

Amazing nights

I just had one of the most amazing evenings I have ever had in my entire life.

There is a very shy, cute woman I know through Salsa. She started perhaps 5 months ago, and I had been caught by her almost since she started.

I remember sitting there with a buddy getting high quite a long time ago. We were watching some indie flick, and just talking about random stuff. I told him that I was very attracted to this beautiful woman, Katie. He said that she was a very classy girl and told me who he thought was pretty cute; I can't remember who he was talking about.

At that moment, while I was fairly stoned, I remember feeling trapped thinking about how much I really *really* was into this girl. I had never really spoken to her, but we had some really amazing dances. Every time I tried to speak to her I just couldn't say anything that made any sense. I remember distinctly my intense focus on her and the tight feeling in my chest regarding her.

The funny thing is that when I am high I am more sensitive than at any other time. Things hurt when I am high that do not hurt normally, and I am open to ideas and random things I would normally not be very open to. Thinking about her when I am in such a state is a sure road to disaster, clearly. I tried to distance myself from such thoughts later because I felt that being in such a state would destroy any chances I had of actually dating her.

Anyway, we danced a couple really hot evenings, and we were getting closer. Finally, at the end of one evening, I pulled her aside and said something to this effect:
"I feel that we are getting very close while dancing. I think there is more to this than a dance, and I want to take this relationship off the floor." She replied that she was interested in the same, but said that it was very important to her to take it slow. I respect that, and I asked for her number and that was it. We held hands for a short while, but there was no kissing or anything like that.

Well, fast forward two weeks and it is time for the Denver art walk. All the galleries are open, there are several artists I enjoy seeing, and there is plenty of wine for any occasion. Of course, when I want to spend a night looking at amazing pieces of art (and some not so amazing...) I am going to try to spend it with the most beautiful woman in the world, right?

And what an evening it was. We got down to Denver around 7, or 7:15. I don't remember much, other than that she really liked to spend time with various pieces. Once she started to focus on some artwork, she took it in and thought about it for a while.

Gradually, as the night wore on, we had more wine and looked at more and more art. The final place we went was an artist's workshop. I know the artist, I have followed him, and I really like his work. Katie, Kyle (the artist) and I spoke at length about his pieces and what it was like being an artist (how many prints did he sell, how much work did he do for commission, etc.).

All the while, I had this amazingly beautiful woman by my side checking things out. She is very shy. When I say very, I mean like so shy when I kiss her hand she blushes and looks down. When we dance, she won't look at me very much. Sometimes I have to sort of kneel down and get under her to catch her eye; she will then look at me for a bit, get a huge smile on her face, and then look away.

Another odd fact is that when I hold her close I cannot concentrate. I almost lost a pullover tonight, because I put it down to dance with her and then I didn't remember it for about 10 minutes. We had to wander back and find it again. I noticed this during salsa; when I just pull her body next to mine it feels like I cannot breathe and like I can breathe for the first time in my life all at the same time. I then loose track of the music, rhythm, and everything else all at the same time.

Finally we went back to her apartment and watched a documentary on children of prostitutes in Bombay. It was very moving and I loved it. Plus I had the most beautiful woman in the world in my arms during the documentary.

It was an amazing evening. We kissed for the first time. What else can I say?

Chris