Category: braydix

LightDesktop Valentine’s Day Release – now Self-Hosting (ish)!

So – biggest thing is that I’ve replaced the horribly-performing WebDAV with a CREST-fs derived writable filesystem. It’s nice, writing both the server and client sides of a remote filesystem setup. I’m enjoying it, when I am not feeling like tearing my hair out. Because I am a fearful little wretch of a man, I did not write the ‘DELETE’ method yet. I’ll get there, just have to build up the courage. I think I wrote the client side, I just want to do something super wussy on the server end where I don’t delete it ‘just in case’…The new filesystem should have several attributes the old one did not have – it should work disconnectedly, it should be cache-aggressive, it should use the cache when disconnected, it should perform well, and it should have some support for symlinks.

I also modified the ‘root window manager’ thing – it has a sorta taskbar style doodad, with a wireless strength meter and a battery power meter, and some little quicklaunchey button things. Not configurable yet, but that’s probably one of the things that’s next.

I started by making it so that I’m able to compile a relatively simple project, with the storage all living in the Cloud. That compile was actually the root window manager/launcher thing for LightDesktop. Then I moved my LightDesktop dev directory over to a separate filesystem. And then booted up into LightDesktop, with that filesystem available as a secondary FS. Took me a while, but I got to where I can do a full compile, rsync, upload – everything – all from LD itself. I’m a little too afraid to put the entire compile for LD into the cloud itself – lots of stat() calls on lots of files causes lots of traffic, and the performance hit is a bit rough right now. But I’m inching my way there!

So, performance still is a little rocky – there are optimizations to be made on my end in plenty of places, I’m sure. I every now and then run into a weird problem where the system hangs on boot at some point after mounting CREST-fs. And lots of networking changes will occasionally confuse the filesystem to the point where it doesn’t work. I really want it to fail over to the cache when it can’t read something, but the allegedly time-out’ed recv/send calls seem to sometimes not work that way. Or maybe it’s my DNS stuff. If you’ve ever done any Unix network programming, I can assure you it is unnecessarily unpleasant, almost all the time. Just going from name to IP is like a page of code, it’s horrible. I see why every reasonably-sized app builds its own DNS cache every time, you can get stuck in DNS lookup hell real quick.

So at this point you could theoretically grab a copy of this thing, boot from it, and develop up a storm on it, building apps for it. I guess I’m kinda doing that now – though my source code is not on the Cloudy CREST-fs filesystem due to my fear of somehow blowing up all of my data (not completely unfounded of course…). But at least from here on out when I’m developing LightDesktop, I’ll at least be running it. That’s a start. I’m not at the point where I can ‘develop from anywhere’ until I can get the source into the cloud too – but that’s not far off. That will be pretty cool – I’ll sit on a VM on my Mac, or on Craptop, or maybe some new Desktop machine I get…or booted off a thumbdrive somewhere. And coding. How cool would that be?

Braydix is now LightDesktop

So I have new downloads of the Braydix thing – I’ve been able to shave boot times way down with the new 2.6.30.x series kernel. I’ve also simplified and stripped down the bootstrap system to help speed the boot times further.

While I was tearing everything up and smooshing it back together, I also took the opportunity to start renaming it to the name I think I’ll be using going forward: LightDesktop. I made a website too (well, I repurposed a botched previous website I made at least…). I actually enjoyed making the text on the website because I went with a far more informal tone for fellow nerds – the only ones likely to ever look at or use the thing.

I’m still unhappy with how it works on my crappy little Asus netbook, but it’s pretty solid on Craptop (my Toshiba). I think there’s something funny with the wireless drivers for the ath5k, but I’m too exhausted to look into it much.

Downloads are up: ISO and FAT fs image for USB memory sticks.

Braydix vs. Google Chrome OS

So I won’t lie – when reports started coming out about Google’s new Chrome OS, I was completely devastated. I liked to pretend I was making Braydix ‘just for fun’, but I think I had always secretly hoped that I might someday make a product of some sort out of it. Google’s announcement basically means the chances of that are nearly completely shot. That did not make me happy, to say the least.

But it’s been some time, and I’ve cogitated on it and pondered and moped, and I think I’m going to keep messing with what I’ve got. There are a few possible outcomes

  1. Google claims their software will be open source. So there’s nothing preventing me from taking whatever I want from that and integrating it with whatever I’ve got. I’ve got a pretty nifty filesystem, and a pretty tight high-level architecture for stuff.
  2. Or, if they really do a great job (very likely), I could just switch to it, completely. Whatever services or things I make available to my ‘thing’ can probably be made to be just as available to the Googlers.
  3. Or if their licensing terms are horrible, or their product is Big Brother-ey, I become the ‘friendlier’ alternative.
  4. Or maybe Braydix just becomes irrelevant.

The likelihood is that Chrome OS will suck whatever amount of developer-juice and hobbyist-interest away from me, and to them. I don’t foresee that changing.

But whatever the GOOG does, they cannot take away from me what I’ve learned, or what I’ve built, and if they make something that I don’t like, I can always make sure that I am making something I do like.

I do think the amount of time I spend on Braydix will decrease. It’s natural – when I have something that isn’t so unique and wonderful and clever as I thought it was, the impetus to work on it decreases. And when Google is advancing the same thing as I am, their bandwagon is a touch bigger than mine.

All that being said, though, here’s a nifty new version of Braydix. It does lots and lots of clever things, but to the end-user just looks like it boots up into a browser, and that’s it. When there are software updates to be had, there’s no ‘updating’ process that has to be run, the new version is just ‘there’. There’s even room for some way of running third-party apps. As a neat little aside, it will even work completely disconnected from the internet…though since all it does is browse the web, there’s not much of a point. Yet. Anyways seems pretty simple, but the truth is far more complicated…

********** WARNING. GEEKS ONLY BELOW THIS LINE ***********

Like any Linux distro, Braydix uses its own initramfs as the tiny bare-minimum memory-based filesystem from which to mount the ‘real’ root filesystem. The Braydix initrd looks for a ‘cache’ hard drive that it can boot from, and boots from it if it can. If it doesn’t find one, it uses a special “bootstrap file” to prepare a minimal set of files that so that it can boot up and launch a browser (and network config utilities and disk config utilities, etc). Once I got the files down to a size where they would fit, I used a ‘CRAMFS’ file to jam them into (it’s a special compressed filesystem).

Once the initramfs has loaded, it passes control on to the cache HD, or the minimal filesystem from the bootstrap. That’s the piece that manages to overmount crestfs over the current ‘web cache’ (whether it be a disk or temp files). CREST-fs is the crazy thing that I built that allows you to mount the whole of the Web as a filesystem. The ‘overmounting’ trick is tougher than it seems – the very files you’re running off of are the ones that are being replaced by a new copy! Gotta be careful – and debugging is a bitch. The cache that it’s overmounted-over is treated as cache, and if the contents are ‘current’, newer versions won’t have to be grabbed from the internet.Then it launches the browser.

The result? a 30 meg ISO. And I haven’t really gotten deep into trying to squeeeeeze and scrunch the files in there down (but I don’t think it will help much). and that 30 meg ISO could have give you access to a huge and complex back-end system – as I am expecting to do. It’s really fun putting things *not* on the bootstrap file, but on the server, and watching them show up on my test VM. That’s probably what’s happening next – write support (somehow), C compiler, and getting it all to be ‘self-hosting’ – instead of using a Redhat install as my ‘workstation’ to develop this, I will use instead a Braydix install to develop it. That will be fun!

Oh, and F google. F ’em in the eye. They’re starting to really freak me out.

The second-to-last release of Braydix…

So I’ve definitely made some Braydix progress – here’s the latest. I need to eventually come up with some kind of name that’s catchier than “Braydix” – that has the syllable for “dicks” in it, which…isn’t that nice. It’s still pretty huge – 140MB right now – but I have things being compiled with debugging information ‘on’ (which bloats the sizes out) and some things that I should have working as libraries are actually compiled-in. I think I may also make some stuff compressed, which very rough back-of-the-envelope math says should cut the size to a third (!).

So what’s in it, and why is it the second to last release? Well, I’ve tied in the CREST-fs ‘internetty’ filesystem pretty early into the boot sequence, so the system in essence can ‘boot off the web’. The idea is, when it’s integrated perfectly, that you would be able to do some kind of initial installation thing, and never have to do any upgrades or anything. Just reboot, and what you need is ‘there’. But right now it’s not quite integrated perfectly – I’m not sure how I do kernel or initramfs updates, or how they’d get picked up by the client. So once I’ve got *that* figured out, then *that* release becomes my ‘final’ release.

I’ve also built a very very simple installer, which you can launch using one of the virtual terminals (Ctrl-Alt-F1 is the main browser window, Ctrl-Alt-F2 through F4 gives you a terminal. Type ‘installer’). I wouldn’t run it on any piece of hardware you cared about though – it could format the wrong disk, or install to the wrong partition, or do any number of other, awful things.

The clever bit – well, there are many clever bits, but the ‘novel’ clever bit – is that I have a special ‘bootstrap’ image to prime the CREST-fs backing cache. So you can, in fact, boot up off this disk image with no network connection at all. And you’ll at least get the browser launched and the GUI up and running. From there, you can possibly configure wireless or something and continue to use the system from there.

The idea – and I don’t think I’ve articulated this very well – is that you should never have to install anything on this. Whenever new things show up on the internet, they’re just ‘there’ automatically. You never have to update anything – reboot and you should have the latest. If I end up doing a third-party application thing, just chuck your binaries and libs and stuff on a website somewhere, and it’s available to the systems – the CREST-fs mounting can talk to any webserver just as easily as any other, mine isn’t particularly special (except for the odd symlink that points to it, or a PATH entry).

Some notes:

  • Libraries SUCK
  • Compiling things ‘static’ is really hard
  • Booting is really complicated
  • The Linux Initramfs is really brilliant. You can do all kinds of ridiculously crazy shit with this. Very cool stuff.
  • Busybox is still awesome
  • “Virtualbox” as a development environment under OS X is (relatively speaking) an extremely pleasant place to develop in

Nerd Alert!

The following post is intended for a very technical audience. Consultant supervision is advised.

I wrote a thingee that lets you mount “The Web” as a filesystem. So far it works under Mac OS X, but I intend to port it to Linux. It uses MacFUSE for the filesystem interface – makes it much easier to write without kernel muckery. It’s all written in C. It’s available on GitHub. It’s called CREST fs – Cached REST, or Cached Representational State Transfer.

The thinking is that The Web, as a whole, is a set of resources, which I should be able to access like files from a file system. So as opposed to doing something like trying to curl http://samplesamp.sa/apage/something, then trying to run the code that lives on that page, you could instead mount the web under some directory, then cd to samplesamp.sa and execute apage/something directly. The first access to ‘something‘ would require it be fetched from across the internet, but with a caching scheme built in, subsequent accesses should be from the local hard-drive caching system.

I thought it would be clever to be able to mount it under a folder called /http:/ – so you could say:
ls -al /http:/www.google.com/someurl/something
See? I think that’s clever. You could probably put two slashes there and it wouldnt’ mess up anything. And if you were sitting in the / directory you could skip the first slash. But that’s starting to get too clever just for cleverness’s sake.

There’s stuff like that Out There already, but I wanted to build something that was extremely aggressive about caching, and very primitive (low-level, usable in Early boot environments). I expect it to notbe remotely coherent, but I do want it to be fast. So far, so good.

Sidenote: it’s my first Git project. Git is nice. Hosting it on GitHub is interesting, too, but less interesting than the fact that it’s on Git.

Writing code in C is painful. Allocating memory is not fun. Troubleshooting subtle memory leaks is not fun. Doing your own string manipulation by hand is not fun. But there is a certain feeling you get from being this close to the bare metal of the hardware…that’s really pretty exciting.

This is the second time I’ve written this – the first version was lost in the Great Hard Drive crash of ’08. Writing all this low-level crap is not that cool, but writing it for the second time is even less cool.

The intent behind all of this is to tie it in with Braydix somehow – to allow you to boot a “minimal” Braydix image from CD or USB key, and have it pull the rest from Teh Intarwubs. I have a new client who does a lot of work within Amazon’s EC2 environment, so I’ve had to study how that works. I think this FS might be interesting for that, too. As soon as I can find an excuse to put something up over there, I definitely will mess around with that too. Once I’ve done that, just think of it – they’ll be Braydix both client and server versions.

Another new Braydix

http://desk.nu/Universix.iso

Now it's starting to get out of the proof-of-concept phase, and into the alpha release stage, almost…

Features:

  • SSL
  • Actually loads images (nice, right?)
  • Should have better VESA/VGA support.
  • (some testing has worked with booting via USB thumb drives, but I don't think this ISO will. Maybe that means I can’t count it as a feature? I dunno.)
  • WIRELESS – using the familiar and intuitive Ctrl-Alt-F2, wpa_gui -something_something, ctrl-alt-F1 method – how easy!
  • Almost installable – if you already have a formatted and correctly
    labelled ext3 partition, you can copy it to your HD.
  • Will run in 128MB, maybe even as little as 64MB.

Uses:

  • Security dorkwads – bad men can't 0wn what isn't 1nst@ll3d
  • Kiosks dweeblings – no GUI other than a browser
  • Old, decrepit POS computer-hoarders – this will actually run on your crappy computer
  • Remorseful netbook owners – this will actually let you do something useful, like checking out Facebook, on your otherwise useless would-be laptop. This is my current use case.
  • Bloat warriors – next time you see a completely horked Windows XP installation or something even worse limping along painfully, know that with 120mb of storage, and 128mb of ram, you could be doing soooo much better…

Unfeatures:

  • Doesn’t have a pleasant installer. Won’t format your drive (though, this could be a plus…)
  • No bookmarks – c’mon, isn’t that what Google’s for?!
  • No disk cache
  • Doesn’t utilize the memory it saves (yet)
  • Doesn’t save wireless settings (by default)
  • Disk image is more bloated than it should be
  • no sounds?
  • No Flash (this is a problem…but solvable?)
  • static IP’s aren’t yet supported (DHCP only)
  • Nasty way of invoking the wireless configurator – and it’s not that friendly

So if 99% of what you do for fun or for work ends up with you browing the web, why have your computer do anything else?

SuperBraydix

So this is the latest n’ greatest of Braydix. Ah, that rhymes, kinda.

Here.

It has bits that look ugly, and the VGA detection isn’t very good, and it only works with Ethernet, but it works for me (not for Bryan, because he did something bad in a past life) on one of my two computers, and in VirtualBox. The font metrics are screwed up for some reason, too. And you can’t actually install it, it only runs off of the CD. It would be a trivial matter to make it run off a USB thumb drive, too, but I haven’t gotten to that yet.

As I mentioned in a previous posting, this is 33 or 34 MB, it’s just got a huge initramfs, and no root filesystem. This makes things soooo much easier. I think it messes with memory usage a bit, but it ran fine on a 256MB machine with, like, a Pentium 2 or some horrible CPU like that. The browser is actually the Qt demo browser – totally untouched by me (to the point that it still boots to a trolltech.com web page).

My next steps are probably to work on making it installable from USB, get better graphics drivers loading up, maybe get some wifi going…maybe fiddle with getting it to launch from /sbin/init, and get it to shut down without complaining. Oh, and get it to load/save bookmarks from Teh IntarWebz. All in due time, perhaps.

Mac OS X good and bad & Braydix w/WebKit

I actually had a slightly not-unpleasant experience with Mac OS X, being used as a Unix server. Since I rail on how much I hate it as a server, I thought I would balance my own bias with a report to the contrary.

I was prepping a mini to run DJB-utilties – DNS, daemontools, Ucspi-tcp. It was not very painful to get them installed. I decided to run my stuff using launchd instead of daemontools, and that worked fine too. I tested it and killed some stuff and it came right back. Also tweaked the Postfix config to use some custom transports, and stay running – easy-peasy. No problems. Found some nice stuff on the internet about how to add users using something like dscl – and that didn’t hurt much at all. The odd piece of dated documentation here and there, but no biggee. Postfix was totally painless, just my weakness with that package that might’ve affected me if only slightly. I should note – and I bet this affects my report – that this was on a regular OS X workstation. When I used to program all day on a Mac OS X workstation which ran apache and php, I also had no problems from that setup.

And now to balance my prior balancing report – I also worked on a Mac OS X Server. And it fucking sucked. Again. Within days of my previous unhappy work with an OS X server – I think it was either Tuesday or last Tuesday – I’m working on another one that’s freaking out, and I can’t tell what’s going on, and the admin tools don’t work, and smoke is pouring out of the side of it, and gear teeth are flying out with sparks. It’s a mess. I keep wanting you to not hurt me, OS X Server, and you always do. Every time. I hate you. It’s like some horrible abusive relationship. I run away, seek counseling, go to a victim-of-abuse-by-os-x-server shelter, get my life together again, and then, months later, I think, “Hey, that OS X Server, he ain’t so bad…it was probably something I did, after all. I bet I can do better.” And then I’m making excuses about how I fell down the stairs and banged my eye into a doorknob again.

I also have redeveloped Braydix (as a necessity). Instead of building it against Firefox, I built it against WebKit, and saved literally 2 or 3 months of work. Holy shit! It’s no wonder that everyone uses WebKit as their base. My God, the difference! I also used the Qt toolkit, and was finally able to get rid of Xwindows, which I am sooooo stoked about. So the latest Braydix uses the Linux framebuffer and bops open a web browser right there. It boots up fast for me on my VirtualBox, but I sent an ISO to good ole Bryan and he wasn’t able to get it to do much without it panicking and dying. So a bit more work to do. Another thing I did was I jammed everything into the ‘initramfs’ and got rid of the whole concept of a root drive. Sooooooo freeing! And the result is a 33 MB ISO (which doesn’t yet work, but shhhhhh…). I’ve been trying to get it to run on this eeePC I have handy but so far, not quite. Just getting Grub on there was enough of a challenge. We’ll see how it goes.

8MB and 10 minutes – proof of a very nerdy concept

I like the idea of booting from a webserver. I’ve always thought this was a cool thing. So I made a new leeeeeetle teeny tiny version of Braydix that actually boots from The Internet. The idea sounds easy, but it’s actually really hard – and it takes forever before it finally finishes booting. I timed it by looking at the server logs and it took 10 minutes before I had a working web browser.

I guess I can go over my decisions backwards – from the end to the start of Boot. I used the latest ISO of Braydix and threw it on a Server to boot from. I used ‘httpfs’ to mount that ISO somewhere, and then mounted that file using a ‘loopback’ option. I had to build a completely custom initrd to do all this with, I started from the CentOS default one and ended up completely scrapping it and building my own from scratch, using Busybox. I grabbed all of the Networking drivers (only!) from /lib/modules and and put them into my new initrd, and I also needed ‘fuse’ (for httpfs) and a filesystem loop module. I made Busybox do the few, measly little things that I needed to get done to be able to mount an ISO – modprobe a network connection, modprobe fuse, a mini-dhcp client, a little teeny shell – it’s great. I have no idea why RedHat bothers with their Nash shell – busybox crushes it. And I used Grub for making my CD bootable – I’m still having problems having it run through the grub.conf automatically, it keeps making me specify stuff. But maybe I’ll fix it and make the ISO URL a boot-time parameter that the user has to put in? That could be interesting.

Anyways, I’ve been poking at this for a week or so, and I just booted for the first time, so I wanted to mention it.

Braydix v0.2

Well, I got some time, and I upgraded my laptop to 4GB, and I finished copying stuff from various other machines I’ve been using, so I got a chance to poke around my Braydix CD project. I was able to clean out a decent number of megs, and fix two…rather crushing bugs.

Bug #1 – reported by Bryan – it doesn’t actually work, at all. So I figured I’d make it work. That seems like a good fix.
Bug #2 – discovered by me, once I made the CD thing work. It uses static nameservers that only work in my house. For people who aren’t in my house, this is probably a good fix.

Issue #1 was actually the stupid VMWare install getting its filthy tendrils into my systems, so it tried to always use vmxnet drivers for the ethernet card. That only works in VMware, not in real life.

I still have some ideas for this thing to make it more ‘interesting’, hopefully nothing so complicated or difficult so that I won’t end up doing it…

So it’s here, I’m still throwing these up on this mediafire thingee because it’s free.