Running wxPython 3.0 on Linux Mint 17.2

I wanted to try out wxPython 3.0 on my machine running Linux Mint 17.2. Unfortunately, Ubuntu 14.04 LTS, that Mint is based on, only has wxPython 2.8 in its repository.

It turned out to be fairly easy to build my own packages for wxPython 3.0 though. Here's a short summary on how to do it.

Download the debian sources for a newer version of Ubuntu:

http://packages.ubuntu.com/source/xenial/wxpython3.0

The files I downloaded were:

wxpython3.0_3.0.2.0+dfsg-1build1.debian.tar.xz
wxpython3.0_3.0.2.0+dfsg-1build1.dsc
wxpython3.0_3.0.2.0+dfsg.orig.tar.xz

But they might be updated when you read this, in that case, modify the commands below to match the current version.

Unpack the sources the same way that "apt source" usually does it:

xzcat wxpython3.0_3.0.2.0+dfsg.orig.tar.xz | tar xvf -
cd wxpython3.0-3.0.2.0.orig
xzcat ../wxpython3.0_3.0.2.0+dfsg-1build1.debian.tar.xz | tar xvf -

Install some tools to build Debian packages:

sudo apt-get install build-essential devscripts

Install some dependencies needed by wxPython itself:

sudo apt-get install python-all python-all-dev libgtk2.0-dev libwxgtk3.0-dev libwxgtk-media3.0-dev

Build the Debian packages. "-us -uc" tells debuild to skip signing the packages, "-b" tells debuild to only build the binary packages:

debuild -us -uc -b

You should now have a bunch of debian packages in the parent directory. On my computer I got the following:

cd ..
ls python-wx*.deb

python-wxgtk-media3.0_3.0.2.0+dfsg-1build1_amd64.deb
python-wxgtk3.0-dev_3.0.2.0+dfsg-1build1_all.deb
python-wxgtk3.0_3.0.2.0+dfsg-1build1_amd64.deb
python-wxtools_3.0.2.0+dfsg-1build1_all.deb
python-wxversion_3.0.2.0+dfsg-1build1_all.deb

All that's left to do is to install them:

sudo dpkg -i python-wx*.deb

One more thing though. This will change the default version of wxPython on your system to 3.0. So any python application that does just does "import wx" will use version 3.0 instead of 2.8:

python
>>> import wx
>>> wx.VERSION_STRING
'3.0.2.0'

This might break some other which did not expect that. I have not noticed any breakage so far, but if you want to be really safe, you can lower the priority of wxPython 3.0 so that 2.8 will be the default instead:

update-alternatives --install /usr/lib/wx/python/wx.pth wx.pth /usr/lib/wx/python/wx3.0.pth 0

python
>>> import wx
>>> wx.VERSION_STRING
'2.8.12.1'

You can then explicitly select wxPython 3.0 in your application using the wxversion module:

python
>>> import wxversion
>>> wxversion.select('3')
>>> import wx
>>> wx.VERSION_STRING
'3.0.2.0'

Now I just have to remember why I wanted wxPython 3.0 on my machine. It'll come back to me soon I hope.

Scratching an itch - rdesktop

I’ve been using rdesktop for quite some time. It’s a nice little program that can connect to a Windows machine using the remote desktop (RDP) protocol so that I can view the Windows screen on my Linux machine.

I’ve mostly been running rdesktop it’s own workspace covering the whole screen. For anyone looking over my shoulder it would look like I’m running Windows, there are no visible traces of the MATE (Gnome) desktop. I turn on keyboard “grab” in rdesktop, which means that if I press any of the key combinations that are normally handled by the Marco (metacity) window manager, they are sent to the Windows machine instead: Alt-TAB switches windows on the Windows machine instead of switching away from rdesktop.

To be able to switch to different desktops it has to be possible to “ungrab” the keyboard somehow. Normally that is done by moving the moue outside of the rdesktop window, but when rdeskop is covering the whole screen that won’t work any more. Instead I added a patch to rdesktop that adds an option “-U” which lets me press Ctrl-Alt to “ungrab” the keyboard. That means that I can press Ctrl-Alt, release the keys and then press Ctrl-Alt-Down to move away from the desktop. It sounds complicated but works really well in practice.

This command line is what I used. “-D” to remove all window decorations such as the title bar and borders, “-U” to turn on my “ungrab” patch and “-g 100%x100%” to make the rdesktop window cover the whole screen.

rdesktop -D -U -g 100%x100% hostname

All this worked really well for a long time. Until something changed in the window manager which made the Gnome panels move to the top which in turn means the Windows system tray would be hidden below the Gnome panels. I didn’t have time to figure how to fix this at the time, instead I started using rdesktop in “workarea” mode where the rdesktop window is sized to the area between the panels instead:

rdesktop -D -U -g workarea hostname

It was really ugly with two panels, both the MATE and the Windows one. It also confused some Windows programs that assumed that the screen must be at least 1024x768 pixels large and on my laptop I would end up with a windows screen which was about 730 pixels high and buttons would end up hidden below the bottom of the screen. But it had to make do.

Using the “-f” flag to get fullscreen mode in rdesktop didn’t work either. If I use that the window is ignored by the window manager and shows up on top of all other windows on all workspaces. Maybe good for a computer in kiosk mode, not as useful when I want a convenient way of switching between the workspace with the Windows desktop and the MATE desktop. The reason rdesktop behaves this way is that it turns on the “override redirect” flag on its window. Override redirect tells the Window manager to totally ignore the window and is intended for short lived windows such as menus or tool-tips.

The proper way to do it, which I just figured out, is to stop using override redirect and set the window property “_NET_WM_STATE_FULLSCREEN” instead. This hint tells the window manager be aware of the window and do the proper thing when switching workspaces, but to otherwise leave the window alone. I’ve made an ugly patch to rdesktop which probably breaks other things, but it works well enough for my use case. So I can now run rdeskop in full screen mode and still have proper workspace switching:

rdesktop -D -U -f hostname

That’s one small itch scratched. Only a couple of hundred left to go.

The maintainer of rdesktop didn’t want to take my ungrab patch when I mailed it to him a long time ago and considering that the fullscreen patch is a hack which probably breaks other things I definitely wouldn’t expect him to take that one. But on the off chance that anyone would find my modified version of rdesktop useful, the source can be found on github.

Kass marknadsföring

Jahopp, nu började jag få mail från netauktion.se igen.  De håller på med oönskade massutskick av mail med reklam för deras auktioner.  Det jag tycker är lite roande är att varje mail avslutas med följande text:

Denna företagsmarknadsföring har sänts till christer@weinigel.se och är avsedd för företagsadresser. Marknadsföringslagen tillåter att skicka marknadsföring till företag - även kallad juridiska personer.

Borde inte det ringa en liten varningsklocka hos dem när de måste ursäkta sig med att det de gör faktiskt inte är olagligt?  Bara den ursäkten visar ju att de vet att oönskade massutskick av email är just oönskat eftersom mottagarna inte har bett om det.

Sen säger de även att man kan avregistera sig och slippa fler mail från dem: 

Önskar du avregistrera dig från framtida utskick, klicka på länken under.  [Klicka här] för att avregistrera denna adress.

Det är en lögn.  Jag klickade på länken för att av avregistrera mig för två månader sen i förhoppning om att de faktiskt skulle hålla sina löften.  Men nej, nu började de skicka mail gen.


Det är ju verkligen jättebra reklam för ett företag att göra sig ovän med sina potentiella kunder.  Min reaktion är att skriva upp dem på listan över företag jag inte vill göra affärer med.  Om de är så oseriösa att de håller på med spam, oönskade massutskick av mail, och ljuger om möjligheten att avregistrera sig, hur ska jag då kunna lita på dem tillräckligt mycket för att göra affärer med dem? Näe.

Netauktion i Göteborg är bara ett av många företag som håller på med sånt här. 

Appspotr, även de i Göteborg envisas med att skicka spam lite då och då.  Varför skulle jag vilja köpa en "företagsapp" av ett företag som spammar?  Om man måste spamma för att få några kunder överhuvudtaget så beror det ofta på att produkten inte är speciellt bra, till exempel så fick deras tjänst en recension i IDGs techworld "Appspotr - trist verktyg för att göra trista appar" - inte så smickrande direkt.  Sen kan man fråga sig hur mycket service man får av ett företag som inte ens klarar av av att hantera sin egen hemsida, letar man rätt på www.appspotr.com så är den helt blank.  Grattis till den som betalt 25 000 kronor för att få en app av dem och sen behöver uppdatera den, ni slapp nog.

Ännu mer tragiskt är att vad jag trodde var seriösa företag som tidningen Dagens Industri håller på och spammar.  När de spammar så lägger de sig på samma nivå som företag som säljer spanska solhus, potensmedicin eller håller på med rena bedrägerier.  De borde verkligen veta bättre.

Getting the HTML source from an Android WebView

On and off I'm working on writing an Atom/RSS feed reader for Android. The feed reader is using an Android WebView to display the contents of a feed and to be able to debug some issues I had I wanted to be able to view the HTML source for a page.

That's should be simple I though, just do WebView.getData() and show it in a dialog. But no, there's no method to get the HTML source from a WebView, so one has to jump through a lot of hoops to do that. Also the examples I found uses WebView.addJavaScriptInterface to do its job which is not such a good idea because addJavaScriptInterface has security problems and doesn't even work on Android 2.3. And I do want my application to run on fairly old Android devices (I still use a Motorola Defy at work, I like the form factor and that it's waterproof).

So after a bit of thinking I figured out an easier way of getting the source. There is a class called WebViewClient which among other things can be used to override what should happen when a follows a link in the WebView. This together with a bit of JavaScript can be used to get the source.

First, when initializing the WebView, tell it to use a custom WebViewClient:

 mWebView = (WebView) mView.findViewById(R.id.web);  
 mWebView.setWebViewClient(new MyWebViewClient());  

When asked to view the source, enable JavaScript for the WebView and then inject a bit of JavaScript which builds and follows an URL containing the HTML:

 public void viewSource() {  
   mWebView.getSettings().setJavaScriptEnabled(true);  
   mWebView.loadUrl(  
     "javascript:this.document.location.href = 'source://' + encodeURI(document.documentElement.outerHTML);");  
 }  

The custom WebViewClient will catch this URL:

 public class MyWebViewClient extends WebViewClient {  
     public boolean shouldOverrideUrlLoading(WebView view, String url) {
         if (url.startsWith("source://")) {
             try {
                 String html = URLDecoder.decode(url, "UTF-8").substring(9);
                 sourceReceived(html);
             } catch (UnsupportedEncodingException e) {
                 Log.e("example", "failed to decode source", e);
             }
             mWebView.getSettings().setJavaScriptEnabled(false);
             return true;
         }
         // For all other links, let the WebView do it's normal thing
         return false;
   }
 }

And we can finally show the source in a dialog:

 private void sourceReceived(String html) {  
   AlertDialog.Builder builder = new AlertDialog.Builder(this);  
   builder.setMessage(html);  
   builder.setTitle("View Source");  
   AlertDialog dialog = builder.create();  
   dialog.show();  
 }  

As simple as that. And it seems to work on everything from Android 2.3 up to Android 4.4.

Although what we get from the WebView does not quite seem to be the source code it was fed to begin with. It seems that the WebView will fix up the HTML so that it is correct and will also decode any entitydefs. So for example if the WebView was fed "<p>Hello W&ouml;rld<p>" what we would get back is "<html><head></head><body><p>Hello Wörld<p></body></html>". It's still useful for what I wanted to do though.

Halloween Milling

A few years ago I bought a cheap CNC mill from China. It's actually a "CNC engraving machine", but it's actually robust enough so that it can mill wood and soft metals such as aluminum. Ḯ've been using it to build electronics prototypes, both for milling PCBs and for milling enclosures.

To control the CNC mill I have been using LinuxCNC (formerly known as EMC2). LinuxCNC takes a toolpath written in G-code which says how the milling tool should move, and translates those into the electronic signals that drive the motors on the mill.

For PCB milling I've been using a tool called pcb2gcode which takes gerber files produced by a PCB CAD program such as Eagle and produces G-code. It works really well and makes it possible to quickly testing simple electronic designs.

For enclosures I have mostly been bying ready-made enclosures and then written a bit of G-code by hand to mill holes in them. I've also been milling wood and plastics for simple stuff such as this lid.

Writinga G-code by hand is a very tedious job, anything more complicated than a few holes requires a lot of work. There are a lot of tools and meta-languages for generating G-code from other languages. I wrote my own little Python library for generating G-code and it turns out that the Axis GUI for LinuxCNC can actually load and run Python programs and execute the G-codes that are printed by the program. But even with all these tools or meta-languages it's still quite tedious to generate the toolpaths by hand.

It would be even nicer if there was a way to create a 3D model in a tool such as OpenSCAD or Blender and then magically let a tool generate G-code, a toolpath which tells the mill how the tool should move to create a physical object which looks like the model. Most 3D modeling tools can export to a standardised interchange format called STL and there are lots of tools which can read STL and generate toolpaths.

Most of these tools are proprietary though and the cost ranges from expensive to insanely expensive. Most of them only run on Windows and I prefer to use Linux and anyway I like open source, so I would prefer to use an open source toolpath generator if possible. There are a bunch of those, but most are at the "alpha" level and nowhere close to being usable. The most mature open source toolpath generator I have found is PyCAM. Unfortunately PyCAM and I don't really get along, I don't like the toolpaths it generates and last time I tried, it was really, really slow. My guess is that one for PyCAM being so slow is that it tries to be very generic and handle models and milling tools of almost any shape. Algorithms to calculate a toolpath for an abitrary 3D shape around an arbitrary 3D model are complex and expensive.

For a long time there have been this thoght at the back of my head that "there must be an easier way". For almost everything I want to mill, I'm going to use a cylindrical tool. With a cylindrical milling tool, the toolpath generation problem is reduced to 2.5D, that is, for each position X and Y the tool is allowed to reach a certain depth Z. This 2.D problem ought to be a lot easier to solve than the generic 3D problem.

A fairly simple strategy of milling 2.5D shapes would be to slice up the 3D model into 2D layers. The algorithm would then start from the top, and remove material for each 2D layer until there is no more material to remove. This is called waterline machining. It might not be the fastest way to things, but it should be fairly foolproof.

For removing material there are lots of ways to do it. The simplest way is to just move back and forth in a zig-zag pattern. But there are much better ways to do it, to reduce the wear on the milling tool it's preferable to keep a constant load on the tool, i.e. cut with the same side of the tool at the same depth and at the same speed all the time. The most important thing is to avoid is rapid load changes such as cutting 90 degrees into a wal. This article att BobCAD-CAM shows some of the toolpah options that used. Option 3, the High Speed Machining (HSM) toolpath, is the best since it avoids any sharp corners and each cut is very smooth.

I have been toying with algorithms like these for a few years but have never gotten anywhere. But a few weeks ago just before Halloween I tried a slightly different approach. The goal was to find an algorithm which would remove material from a 2D outline and do it using the same side of the tool to perform all cuts and which would try to keep the cuts at a constant width.

What I did was to split the toolpath generation into four steps.

1. Take the model and shrink it by the milling tool radius. This way the algorithm can ignore the tool with for the rest of the calculations, instead it only have think about where the tool can move.

2. Find the center of largest circle that can be inscribed in the model. To do this just shrink the model until there is nothing left and use the center of the last polygon as the starting point.

3. Start cutting a spiral from the starting point outwards until the spiral intersects the outline. For each 360 degree turn the radius of the spiral grows by the cut width.

4. Cut away the rest of the material. Start by creating a polygon with the the material that has already been cut away, this will roughly be the convex hull of the spiral. Grow the polygon by the cut width and then clip it with the model. The difference between the polygon of already removed material and the grown and clipped polygon are a number of polygons with material that is possible to remove. Choose one of them, mill away the material, add the polygon of just removed material to the big polygon of removed material and start over again until there is no more material to remove.

I started out by using the Python port of the Clipper library to do the shrinking, growing and clipping but switched to Shapely after a while because I needed some function that wasn't available in Clipper. A function that I ended up not using in the end anyway, but by then I couldn't be bothered to switch back.

This actually turned out to work quite well. Because it was Halloween I decided to do a small test with a witch silhouette I found. I converted it by hand from SVG to a bunch of coordinates (and managed to flip the image horizontally and lose a few parts of the broom on the way).

The following video shows the algorithm running.

Since this seemed to work so well I quickly added some functions to generate G-code and then used LinuxCNC to run the G-code on the mill. And what do you know, it actually worked quite well.

The video is speeded up 10 times, in real life it took about 20 minutes to run.  The algorithm video and the actual milling video differ a bit in the cuts since I tweaked the algorithm slightly in between.

Of course this is just a quick proof of concept hack to see if the algorithm would work at fall, but it does look rather promising.

There are lots of things to do though to bang this into shape so that it is actually useful.

The algorithm makes a few mistakes where removes too much material at the same time. It really should try to find a way of smoothly moving into each cut.

The algorithm does not support multiple pockets or pockets with holes.

The algorithm only cuts one layer in 2D, to be useful for milling it needs to be extended to 2.5D.

Yet another pre alpha not even close to production ready piece of software for generating toolpaths. Probably not that interesting, but if anyone wants to see the source code, drop me a line.

Lifehacking

I'm a nerd, I find technology fascinating. In my career I've done just about everything with computers, from being a system administrator changing disk packs for backups to writing J2EE code or designing and building simple electronics. And just about everywhere I've seen fascinating stuff that I would like to learn more about and have had ideas about interesting things that I'd like to try out. But over the last few years I have been working full time or more and really have not had the time nor energy to do that.

So I've decided to make a bit of an experiment with my life. A few months ago one of the customer projects I was working on was coming to an end, and instead of getting a new full time contract I decided to try to work part time, one week at a customer and then one week on my own. That didn't work out too well, as soon as some former colleagues heard I was free they convinced me to work on another customer project, so I ended up working full time again. Oops. Anyway, now that project has ended, so it's time to try again.

So what am I going to do with this week on my own?

Well, one thing I'll do is to try to spread the word that I'm available for hire if anyone has any interesting problems they want solved. I like really hard troubleshooting under time pressure, a "mission impossible" in the borderland between hardware and software, for example where a customer has a problem with a device driver or new piece of hardware that just won't work reliably and the deadline is approaching rapidly. I've actually had some success in solving a few problems like this where the customer didn't know if it was a problem with the hardware or the software and was running out of ideas.

“In the fields of observation luck favors only the prepared mind.“ – Louis Pasteur

I don't always succeed, but somehow I'm quite often "lucky" and usually manage to solve the problem or at least identify the cause.

But I probably can't count on getting these kinds of jobs all the time, so I'll probably have a lot of free time on my hands. I'll try to dig up some of my old ideas and see if I can do something with them. Or maybe just spend some time writing about them on my blog and see if someone else might get inspired and do something.

One such idea I've had since Google Reader was shut down is to write my own feed reader. And when I finally did get around to it, it didn't take much more than a week to actually write something that worked. The result is Weader, a simple atom/rss feed reader for Android. This was all about scratching an itch, I didn't like any of the existing feed readers out there, so I wrote one that works the way I wanted. And it also means that I got up to speed with writing Android applications, something that I've been meaning to do for a long time.

I'll probably run out of money long before I run out of ideas, so my plan is to try this out for a few months, after that I'll probably have to go back to working full time, but who knows, it might work out. And if anyone has any interesting problems that need solving and want to hire me, drop me a line. My rates are negotiable.

Creating Android menu icons with Inkscape

I've spent a few weeks writing a feed reader for Android, Weader and wanted to have nice little menu icons for every possible choice.  Android comes with a bunch of standard icons in the SDK and it's probably a good idea to use them to begin with so that an application will fit the standard Android look.  But I needed a few more icons.

I tried to find some instructions about how to create icons that look like the standard menu icons but the information on the Android developer pages about iconography was rather brief and in some cases flat out wrong.  The offical pages say that icons meant for a black background should be use color #ffffff with 80% opacity which is just plain wrong, if you try to do that any white parts will end up being too bright and what to do about black areas in your icons?

After a bit of reverse engineering of the existing icons, writing a bit of Python Imaging Library code to extract the alpha layer, poking at the pictures with Gimp and a lot of fiddling around in Inkscape I finally managed to produce a couple of icons that I'm fairly happy with.  So to spare anyone else the same problems, here's how I finally generated my icons.

Icons in Android must be PNG files and they should be provided in multiple sizes to fit devices with different screen resolutions.  The "active area" is the area that should be covered by the actual objects in the icon, there is supposed to be a bit of empty space around.

Resolution	DPI	Menu Icon Size	Active Area
ldpi	120	24x24	18x18
mdpi	160	32x32	24x24
hdpi	240	48x48	36x36
xhdpi	320	64x64	48x48
xxhdpi	480	96x96	72x72

Actually, the ldpi icons in Android seem to be 36x36 which is strange since if I calculate the proper size based on the DPI it comes out as 24x24.  Anyway nobody uses ldpi any more so I'll just ignore that and use 24x24.

Inkscape's native file format is SVG, Scalable Vector Graphics, which is an open standard for vector graphics, so that is the format I'll use.  The resolution for SVG files is hardcoded to 90 DPI (dots per inch).  This doesn't match the Android resolutions very well, I just decided to pretend I did everything in the hdpi resolution then fix things ups when exporting an image in PNG format later on.

So, here's what I did.

First I created a new image in Inkscape and then selected File -> Document Properties and set the image size to 48x48 pixels to match the menu icon size for the Android hdpi resolution:

To make things a bit easier for me I switched to the Grid tab and set up a 1x1 pixel grid with major lines every 8 pixels.  Time to start drawing:

I needed quite a bit of experimentation to get about the same line weight and color as the stock Android icons, but finally managed to get something that looks decent.

Select the Object -> Fill And Stroke menu.  Set the fill to white (#ffffff) and the stroke to black (#000000) and then select a stroke width of 1.25 pixels and a blur factor of 2.5.

The final touch is to set the layer opacity to 60%.

Export the drawing using File -> Export Bitmap.  First press the Page button to use the page boundaries as the limits for the bitmap and then set the size to the icon size you want, in my case I chose 48x48 pixels for hdpi. And then press export to create the bitmap.

It's possible to use Inkscape from the command line to automate the creation of the PNG files.  Put the SVG file in a directory called "art" in the project and then run the following shell script:

while read dir dpi; do
inkscape 
    --export-area-page --export-dpi=$dpi \
    --export-png=$dir/ic_check.png ic_check.svg
done <<EOF
../res/drawable-ldpi 45
../res/drawable-mdpi 60
../res/drawable-hdpi 90
../res/drawable-xhdpi 120
EOF

The trick here is to use the correct resolution.  Since the SVG file is in 90 DPI and is drawn for the hdpi resolution, I just have to scale the requested resolution to get the correct PNG image size.  This trick also means that I an use the same script for my launcher icon which is supposed to be 72x72 pixels in hdpi.  Draw it at that size in Inkscape and it will automatically be exported at the correct sizes for a launcher icon. 

So here's the result, ldpi, mdpi, hdpi and xhdpi icons.  The top row are my new icons, the bottom row are the original Android icons.

The other icons are very, not perfect, but they'll do.  I am a bit confused though, the image here was composed in Gimp and the colors look quite different.  When I tested this on an actual Android device the color for the white part of the icon was identical.   And as you can see, the ldpi icon in stock android is completely different, it's using the pre-Honeycomb style for icons, but I'll ignore that and use the same theme for all different resolutions of my icons.

Trying to create an icon which is identical to an existing android icon is probably a bit unnecessary, but the nice thing about having a nice little icon in SVG format is that it's very easy to modify it to create the extra icons I needed in my application, all with the same look and feel.

By the way, as you can probably guess, the icons are used to mark articles as read or unread.  I'm not totally happy with these symbols, the check mark and the cross don't feel right, on the other hand I can't come up with any better ones.  If you have any ideas for icons that might fit better, please drop me a line, I'm always open for suggestions.