G-Man's Uber Software Engineering Blog

Safari Books Online

2011-11-10T15:54:00.000-08:00

I always loved the idea of having access to the Safari library and I had recently another couple of months paired with some time to actually use it. Again despite trying to read as much as I could I only managed to read about three to four books in the last four months. I enjoyed it the most on my 24" LCD monitor followed by the laptop scree and last by my TouchPad. I was running the TouchPad on both WebOS and Android but the browser version just stinks. Make some dedicated reading app - That can't be too difficult. Another note on the TouchPad: I read the Economist (Safari doesn't have a usable offline mode) on an airplane ride and my eyes hurt after two hours.

The missing offline mode supposedly gets mitigated by earning tokens which you can use to download content (like one token per chapter) but after three months I didn't earn enough tokens to download any book which renders this feature completely useless.

The interface is borderline useless. The HTML interface is worse than the real one but I am still trying to figure out how to effectively get to the next page without using the scroll bar. A web browser does this with but that doesn't work seamlessly in Safari. As much as I like it when a book looks like a book - a screen doesn't have pages I need to turn. The chapter should just be a flow of text and NOT interrupted by footers and headers...

I actually did the math and if I would have to pay myself for books I might be better off just buying a Kindle version every month than subscribing to Safari. So as much as I like it and as much as I would want every company I work for to give me a subscription it isn't that valuable for me to get a subscription myself if the workplace doesn't offer it. In that case I would rather by the books outright...

Play on Heroku

2011-10-25T12:20:00.000-07:00

Having some time on my hand I finally got to check out play on Heroku. I have been dabbling with play earlier when I was trying to run Scala on it but this time I was trying to do some barebone Java app. For a chance from usual dev environment eclipse I was taking IntelliJ for a spin.

As opposed to the description here you need to say under VM args

-Dapplication.path=“path to your app”

The "." is just wrong and wouldn't work for me -- I guess you would need to start IntelliJ in the same directory your play app is located?

Anyhow, development was pretty straight forward. I enjoyed that heroku was forcing you to use git and the deployment was also very easy (compared to GAE).

s+b Generation C

2011-08-01T10:43:00.000-07:00

I recently noticed an article on s+b about "Generation C" - which highlights some of the ideas I have been teaching and speaking about. Though they came up with that independently from me it just shows that great minds think alike :-)

Speaking @ SoCalCodeCamp

2011-06-25T21:23:00.000-07:00

The link to my presentation on Evernote.

Book Review: Career Game & Power

2011-06-03T14:31:00.000-07:00

I have read a couple of career books before and they mostly were motivational or geared towards getting a job so I was intrigued to find some book which uses game theory ("Your Career Game") to help you advance and land a job. Unfortunately the book is written mostly for MBA students so the game theory part is surfacy at best and if you are used to at least engineering math kind of disappointing. Nevertheless they finish each chapter with interviews of real people and highlight what "career game moves" they did.

The essence of this book is basically to find a sponsor and mentor who takes you under his wing. In my career I was fortunate enough to have both when I started out at PwC after school. My boss was my mentor and he was great and put me in touch with other interesting people in the organization. Things just happened naturally without much thinking for me. I guess because the main goal was to bring in money (by billing hours or selling projects) there was no ambiguity and it was in my bosses best interest to expose me to many people to get me billing more hours up. Promoting was good again because then they could charge more for me. Needless to say the partnerships goal were pretty much aligned with my career.

When I came to the US I started working in software engineering and had a hard time finding bosses who acted as sponsors or mentors. Honestly I still thought things would just happen naturally but I guess in that case the first book ("Career Game") would recommend to do better homework if the company you are considering is "good for your career" having a reputation for great mentoring or sponsoring.

I liked the second book "Power" much more because it's lessons are far more applicable. They show research that for instance the performance rating is totally unrelated to your performance and more to how much your boss likes you. Consequently you should spend your time managing up and if you made a bad first impression just leave. Pfeffer also gives tips on building a power base (by choosing influential mentors) and cites surprising research like over-the-top flattery is not considered insincere by the recipient.

They are both pretty good books because they prompted me to some action. The first one ran in some open door with the mentoring: I totally need to increase the numbers of mentors in my life. But I think the "Power" book will be more helpful because I should probably focus more on building a "power base" and increasing my "likability".

Working with four continents

2011-03-30T13:43:00.000-07:00

The last couple of months I have been working with a team out of Israel, a team out of Shanghai, an architect based in London, and me out of the US.

My job is to work under the guidance of the architect to develop a component which integrates tightly with what the team in Israel is doing but keeping the people in Shanghai in the loop so they can take over my piece. The easiest part is to keep Shanghai in the loop - I just do a presentation/training session every three weeks in the evening (usually 5 pm my time). They listen, ask a few questions -- but from colleagues having done such things I know that it might be good to give them homework and quizzes to assess the level of their understanding. But I kind of refuse (like I dow with my adult students) to drop people back to college level :-)

Working with the architect is awesome, too: I have him on IM and can drop him a note and he gets back to me really fast. But since he is like me new to the Israel team he does know the same amount as I do about how things should be done on this micro code level where teams differ. Do you use return codes for errors or exceptions? (Turns out they do both) and so on.

I am mostly working with a Senior Engineer and a manager in Israel and they are both really sharp people and a pleasure to work with. I send them e-mails and they usually get back the next day. I send them diffs and they tell me what to change and we have frequent screen sharing meetings (7 am my time) to discuss some of my more complicated changes.

That all sounds great until you discover at 9 am some pattern or detail in their code you don't understand but you probably need to to do your work. Then you have to wait until the next day for the answer. So I found it helpful to switch to some other thing to work on if I am blocked or do some speculative coding to show them how I think it should be done and send it to them to correct me. I have discovered quite some things that way which weren't apparent (to me) by reading their code.

Does it make sense to work that way? I am pretty convinced if I would be co-located at least with the Israel team I would avoid some rework, spent less time searching their source code, and ultimately get more done. On the downside I would probably slow them down a lot with me asking them stuff every 10 minutes or so.

On the other hand I know the piece I am supposed to code for them inside out so having them do it would probably take even longer. But all things equal having a Senior guy on site might be more productive.

I always advocate to fly people to actually meet each other for a few days to code together and get to know each other but with travel budgets tight that often isn't an option. Also often teams contain people from many continents so the way I work this though second best is probably more productive that other things.

The management of my coding is old school. So I don't call in to any daily stand-ups nor do I participate in planning or review meetings. I just keep a representative on each continent informed about my progress and they coordinate with QA and other parts on my behalf. Which let's me focus on coding and I don't have to get on the phone at crazy hours.

I know that this is frowned upon in the trade and we all should have one big scrum we all call in together but this works surprisingly well -- which I attribute to the fact that most of the people I interact with are quite senior and just know what to do.

Play framework with cygwin

2011-01-19T15:23:00.001-08:00

For whatever reason the people doing the play framework force people on Windows to use cmd instead of cygwin. If you have the Python installed under Windows (not cygwin - "the windows and the cygwin python are completely different" - http://cygwin.com/ml/cygwin/2004-02/msg01120.html) you can use the following script. I name it play.sh and put it in the play directory:

#!/bin/bash play=`which play` python `cygpath -wp $play $1 $2 $3 $4 $5 $6 $7 $8 $9`

Play attention to the quotes... (the script is based on the idea put forth here: http://www.inonit.com/cygwin/faq/)

Note: You still need to have the play directory in your path otherwise which won't work.

Membase talk

2011-01-18T09:58:00.000-08:00

Yesterday I had the pleasure to listen to a talk from Membase. They lost the T-Shirts in the mail, didn't have beer with the pizza - but promised to go to a non-specified bar afterward. I left after about 3.5 hours and they still weren't on the way to the bar...

Membase is basically the popular memcached with a persistence layer. It has dual licensing - once a "community edition" under Apache license and another one where you pay. They are used on hundreds of servers at Zynga and NHN. Membase is protocol compatible to memcached but it's using moxi (a memcached proxy) for scalability and potential failover.

Membase doesn't have many of the fancy features other No-SQL databases offer. They don't have a query language nor some map-reduce and even no automatic failover. In fact a node will confirm that it has stored a value before it distributes it to other nodes and writes it to disk. Which leaves the data in case of failure exposed to some minor loss or at least inconsistence (aka the latest value might be gone). Furthermore in the failure case the cluster only notifies that a node failed so clients will get errors accessing the failed node until a system administrator has decided manually a failover is apporiate (which is clicking a button on the UI) or brings the broken node back up. They were also decidedly mute about network splits (one of the big enemies of distributed system) and don't support multi-datacenter deployments (the most common source of those splits) yet.

They are basically some memcached on steroids, fast, highly distributable, (kind of) consistent, etc. They don't solve the same problems like dynamo or cassandra who either allow you to always write and/or always read from the data store at the expense of consistency. Membase overall is highly consistent (except in some error cases) at the expense of failing reads and writes once a node dies.

The administration, set-up, etc. are really easy and being able to use the existing memcached protocol makes this a clear winner in the ease of use. They also have some interesting features with the "tap-interface" which allows third party modules to look (aka tap) into all the data in the cluster and do smart things with it. An example would be lucene indexing...

The use cases are either an extension of memcached (e.g. session caching) and/or something which needs massive scale, speed, and simplicity like Zynga's games. One mantra they repeated over and over was: "If half our servers fail we want to be able to handle half of our user's requests" Not every system scales that way so this is quite an accomplishment.

Future plans include to get some more of the popular No-SQL features like (prefix and range) queries and code running on the nodes (like map-reduce) called "node code".

Given the real potential of data loss this system probably appeals to people already using memcached and running out of memory. They now can add disk storage to it together with some very cool administration ui. There are some other use cases where the eventual loss of data is outweigh by ease of use and speed. Apparently gaming and entertainment can tolerate this risk whereas I wouldn't recommend that solution to store bank account data ;-)

Seattle CloudCamp

2010-12-02T13:50:00.000-08:00

To learn more about the cloud (after all I will be teaching a cloud class in about a month) I went to CloudCamp here in Seattle last night.

They had speakers from all the major companies: Amazon Web Services, Microsoft Azure, and Google AppEngine. Here are my brief notes:

Cloud Security seems to be the hot topic du jour and the way to deal with it is encryption to keep data safe and digital signatures to make sure nobody forged the data

IBM announced another appliance which disperses web sphere images to hyper visors – not sure why they always invent new 1U appliances...

Google was there in force and was pushing their appengine (they now have some way to use their machine learning algorithms in the cloud) but the most interesting thing was AppScale an OpenSource technology to install an AppEngine environment on your own servers

Microsoft gave a keynote about their Azure thing which seems to be shaping up. Anyway, they hyped AppFabric as the solution to everything --

Disaster Recovery in the cloud: Amazon is offering a Virtual Private Clou d to replicate a company’s physical infrastructure and use it for disaster recovery just in case. I believe this will have a significant impact on continuity planning and is something to watch.

Community Cloud – companies/universities/etc. consolidate their computing centers to have one. As example they cited some site in China where in some technology park the provincial government (together with IBM) was running some data center used by all the tenants of this park.

The speakers agreed that NoSQL is going through some big fragmentation – right now we have >40 NoSQL solution whereas there are only 5-6 SQL databases. This needs to be weeded down to a sustainable amount…

Another thing they agreed on was that there is no clear, good definition what the cloud actually is

NoSQL's Tim Anglade in Seattle

2010-09-30T09:02:00.000-07:00

You know that a meeting is important if a friend from San Diego forwards you a meeting announcement for Seattle. Tim Anglade is the initiator of the NoSQL Summer which I actually attended a few times here in Seattle. Tim is on a trip around the world to videotape interviews with all the important NoSQL guys... interesting task and probably a great job if you can get it ;-) In his presentation he talked about the following topics:

Open source or not?
He thinks that Open Source might lead to compromises in quality since everybody can add whatever they want. More importantly he believes though NoSQL systems are complicated and need plenty of support right now this might not hold true for the future as companies build up capabilities in house. Hence the Open Source business model might not be sustainable for the NoSQL companies in the long run.

Fanboys ahoy
Most people like one NoSQL implementation, e.g. Couch vs. Mongo, over another without properly looking at strength and weaknesses of each. I guess what he wanted to say that if your only tool is a hammer everything looks like a nail. So it seems to be important to take a step back and look for the best NoSQL solution for a given problem rather than the other way around.

Is it an industry yet?
Most definitely not but they are getting there. A wildcard in Tim's view is if some heavyweight like Oracle is getting in. This would validate the technology and accelerate growth...

Nobody cares?
He through up a slide from a recent eWeek survey where about 44% didn't care about NoSQL. My impression is that NoSQL is gaining traction a lot of places and will be quite relevant in developing new products. I am not sure how long it will take to trickle down to everyday operations -- will your SAP run on NoSQL next year? Probably not...

Where's the ceiling?
With the database world growing at a rapid pace (there is more and more data each year) the market is getting bigger and hence the slice for NoSQL is getting bigger (by just growing with the market). So the jury is still out there if it is a 100 Mio market or several billions....

Consolidation or Balkanization
As he pointed out in the fanboy thought the NoSQL community is torn between fans of one project over another which suggests Balkanization. On the other hand projects could take ideas from other projects and incorporate them (to even merging - like Struts 2 attempted for the web). I have seen a lot of Balkanization myself but with the NoSQL summer a little bit more understanding of other technologies seems to happen.

Lowering the bar
NoSQL is hard to understand so he is urging all the projects to "lower the bar" by providing good documentation and example configuration files which allow developers to have a system up and running in minutes...

He is also hoping that his efforts like the NoSQL summer will foster understanding of the technology and therefore "lower the bar".

Some new stuff he has planned are the NoSQL tapes and the NoSQL winter.

Palm Pre Plus on TMobile

2010-08-10T21:43:00.000-07:00

A few weeks back at the Capitol Hill Blockparty I put my name in a raffle for an ATT branded Palm Pre Plus and a few days later I got an e-mail from Palm letting me know that I won a spanking new one.

When it arrived it was of course sim locked which means it didn't work with T-Mobile. Calling ATT and asking for an unlock code (as the phone instructed me to do) didn't yield any result because the phone wasn't in their data base. The Internet claimed that you can actually buy some sim unlock and make it work with T-Mobile (Edge only). I went ahead and sent some money to the nice people from nextgenserver and promptly received a couple of e-mails.

I installed their software on my Windows 7 computer (this will get important later) but the instructions said that you need an activated phone. Though I discovered later they had some instructions at the end of the document (I never read manuals from start to finish) I started off to unlock my Palm via WiFi on my own and found this great description. Basically you want to install meta doctor by following the description on the web site. But before you doctor your jar you should adjust the make file. Especially some option which lets you bypass the FirstUseApp needs to be activated. I ended up running the meta doctor on my CentOS server (so any Linux will probably do) and then installed on the Palm from my Mac.

This allowed me to activate the device via WiFi and got me one step closer to actually using it. Back to the nextgenserver's software: It didn't work and I asked support - back and forth troubleshooting they finally revealed to me that Windows 7 wasn't supported which was also clearly stated in the manual (did I already say I don't read manuals?). At least I can attest their support is awesome.

Then I remembered that I had an old WinXP installation running under VirtualBox on my Mac. I put nextgenserver's software there, made it that the Palm was handled by VirtualBox (just click on the USB icon after you connect the Palm), and then after a couple tries (for whatever reason nextgenserver's software doesn't work the first, second, ... time) I finally got my unlock code. Being smarter I finally read the manual and found the part about resetting the diag drivers so I can put mp3s on the palm with USB. Did that and now it works like a charm.

I don't know if that is because I run it on T-Mobile but I noticed:

It drops calls. Never happened with my Blackberry.
It crashes when I put it in my pocket (this seems to be a common problem and can be solved with some paper)
It get unresponsive very often - the Blackberry showed a spinning wheel but the Palm just is unresponsive so I don't know what is going on.
The edge data doesn't seem to be happy all the time. But that can also be TMobile. I occasionally had to reboot my Blackberry to get the data back. (might be TMobile's revenge for me being on a grandfathered $5.99 data plan)
The web browser is awesome but e-mail lacks a search function. The GMail app on Blackberry had that and I am addicted to it. But the Palm mail looks nicer
Almost every app looks nicer on the Palm. TinyTwitter on the Blackberry was a pain whereas Spaz on the Palm is awesome
The card interface is awesome. I really like being able to check the status of each app while sliding
The browser stops rendering when in card view. It will only load the web site when in full view, WTF?? I love opening multiple browser windows... and then checking back. Opera mini on Blackberry did a better job.
The headphones leave an aesthetically unpleasing gap...

It's a really neat device and far more fun then my old Blackberry... so overall I am happy.

32 bit Flash on 64 bit Linux

2010-06-28T11:30:00.000-07:00

I am running some 64 bit version of CentOS and was quite happy for a while with the 64 bit alpha Flash plugin Adobe made available in their labs for some time. Hulu changed from working to non-working, etc. but other sites like ComedyCentral were happy.

Anyway, last week Adobe pulled the 64 bit version of Flash and so I decided in order to get a better Flash experience to install the 32 bit Flash.

I spent about two days on and off playing with the various settings of nspluginwrapper which gave me Flash without sound, Flash which crashed, Flash... but it never worked right. This morning I decided to just remove the 64 bit Firefox and install the 32 bit one instead, added Flash and everything worked as expected.

There are some tricks so just follow those steps:

Add adobe repository to yum (see here - step 2)
Add the 32 bit repo to yum (see here - step 4 b)
Remove firefox - yum remove firefox
and remove flash, nsplugin if installed previously
Add i386 versions of everything: yum install firefox.i386 flash-plugin.i386 alsa-utils.i386
Try it out
If it doesn't work follow the tip here and remove and install alsa libs again.

Hope that makes your flash work...

Talk at SDJUG, 4/20 7 pm

2010-04-14T14:37:00.000-07:00

I will be giving a talk at the local Java User Group about The Server Side Java Conference in Vegas I attended a couple of weeks back. Please check at http://sdjug.org/ for more information.

I will be talking on:

HTML 5 and the Semantic Web

The future of web frameworks

Java EE 6.0
JSF 2.0
Java FX
Spring 3.0

The Cloud

The Cloud Continuum
Elastic Data on the Cloud

This blog has moved

2010-03-29T14:35:00.001-07:00

This blog is now located at http://gman.eichberger.de/.
You will be automatically redirected in 30 seconds, or you may click here.

For feed subscribers, please update your feed subscriptions to
http://gman.eichberger.de/feeds/posts/default.

NIO proxy for manipulating traffic

2010-02-25T10:35:00.000-08:00

I wrote some little proxy for manipulating the traffic between a web service client and server - though it can be used for other traffic as well. I used those two fantastic sites to brush up on NIO: Example Depot and IBM developerWorks. Here's the code I came up with:


import java.io.IOException;
import java.net.InetSocketAddress;
import java.nio.channels.SelectionKey;
import java.nio.channels.Selector;
import java.nio.channels.ServerSocketChannel;
import java.nio.channels.SocketChannel;
import java.util.Iterator;

public class NIOTunnel {
    int port = 8080;
    String hostName = "localhost";
    int hostPort = 8089;
    private ServerSocketChannel ssServerChannel;
    private Selector selector;

    protected SocketChannel createClientSocket() throws IOException {
        SocketChannel sChannel = SocketChannel.open();
        sChannel.configureBlocking(false); // Send a connection request to the
        // server; this method is
        // non-blocking
        sChannel.connect(new InetSocketAddress(hostName, hostPort));

        return sChannel;
    }

    protected void createServerChannel() throws IOException {
        ssServerChannel = ServerSocketChannel.open();
        ssServerChannel.configureBlocking(false);
        ssServerChannel.socket().bind(new InetSocketAddress(port));

    }

    protected void setUpAndWait() throws IOException {
        selector = Selector.open();
        // Register both channels with selector
        createServerChannel();
        ssServerChannel.register(selector, SelectionKey.OP_ACCEPT);

        while (true) { // Wait for an event
            selector.select(); // Get list of selection keys with pending events
            Iterator it = selector.selectedKeys().iterator(); // Process
                                                                            // each
                                                                            // key
            while (it.hasNext()) { // Get the selection key
                SelectionKey key = (SelectionKey) it.next(); // Remove it from
                                                             // the list to
                                                             // indicate that it
                                                             // is being
                                                             // processed
                it.remove(); // Check if it's a connection request
                SocketChannel socket;
                if (key.isAcceptable()) {
                    System.out.println("Acceptable Key");
                    ServerSocketChannel ssc = (ServerSocketChannel) key.channel();
                    socket = (SocketChannel) ssc.accept();
                    socket.configureBlocking(false);
                    ConnectionHandler h = new ConnectionHandler(socket, createClientSocket());
                    h.run();
                }

            }
        }
    }

    public static void main(String[] args) throws IOException {
        NIOTunnel tunnel = new NIOTunnel();
        tunnel.setUpAndWait();
    }
}

public class ConnectionHandler extends Thread {
    private SocketChannel serverChannel;
    private SocketChannel clientChannel;
    private Selector selector;
    private ModifyTraffic modifyTraffic;
   
    public ConnectionHandler(SocketChannel serverChannel, SocketChannel clientChannel) {
        this.serverChannel = serverChannel;
        this.clientChannel = clientChannel;
        this.modifyTraffic = new ModifyTraffic();
    }
   
    public void run() {
        try {
            selector = Selector.open();
            while (!clientChannel.finishConnect()) { 
                //wait
            }
            SelectionKey client = clientChannel.register(selector,SelectionKey.OP_READ|SelectionKey.OP_WRITE);
            SelectionKey server = serverChannel.register(selector,SelectionKey.OP_READ|SelectionKey.OP_WRITE);
           
            client.attach(new WriteStorage(false));
            server.attach(new WriteStorage(true));
           
            while (client.isValid() && server.isValid() )
            {  
                selector.select();
                Set readyKeys = selector.selectedKeys();
                Iterator it = readyKeys.iterator();

                while (it.hasNext()) {
                    SelectionKey key = (SelectionKey)it.next();
                    it.remove();
                   
                    if (key.isValid() && key.isReadable()) {
                        String ret = readMessage(key);
                        WriteStorage ws = (WriteStorage) key.attachment();
                        if (ws.isServer()) {
                            ((WriteStorage)client.attachment()).append(modifyTraffic.server(ret));
                        } else {
                            ((WriteStorage)server.attachment()).append(modifyTraffic.client(ret));
                        }
                        continue;
                    }
                   
                    if (key.isValid() && key.isWritable()) {
                       writeMessage(key, (WriteStorage)key.attachment());
                    }
                   
                }
            }
           
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
            return;
        } finally {
            try {
                selector.close();
                clientChannel.close();
                serverChannel.close();
            } catch (IOException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
        }
    }
   
    public void writeMessage(SelectionKey key, WriteStorage writeStorage) throws IOException
    {
        SocketChannel socket = (SocketChannel)key.channel();
        ByteBuffer buffer = ByteBuffer.wrap(writeStorage.getString().getBytes());
        int nBytes = socket.write(buffer);
        writeStorage.clear();
    }
       


 
    public String readMessage(SelectionKey key) throws IOException
    {
        int nBytes = 0;
        SocketChannel socket = (SocketChannel)key.channel();
        ByteBuffer buf = ByteBuffer.allocate(1024);
            nBytes = socket.read(buf);
            if (nBytes == -1) {
                return null;
            }
            buf.flip();
            Charset charset = Charset.forName("us-ascii");
            CharsetDecoder decoder = charset.newDecoder();
            CharBuffer charBuffer = decoder.decode(buf);
            String result = charBuffer.toString();
            System.out.println(result);
            return result;

    } 
}

public class WriteStorage {
    private StringBuffer toWrite;
    private boolean server;

    public WriteStorage(boolean server) {
        this.setServer(server);
        clear();
    }
    
    public boolean isServer() {
        return server;
    }
    public void setServer(boolean server) {
        this.server = server;
    }
    
    public void clear() {
        toWrite = new StringBuffer();
    }
    
    public void append(String s) {
        toWrite.append(s);
    }
    
    public String getString() {
        return toWrite.toString();
    }   
}

WBem, Wiseman, Windows, Java and Negotiate authentication

2010-01-20T13:43:00.000-08:00

I was playing around with WBem and tried to access the Windows Remote stuff on my Vista laptop. To be special Mocrosoft is calling their WBem implementation Windows Remote (WinRM). Because it's complicate here is some good blog entry on how to make WinRM work for you.

Back to Wiseman: Denis Rachal has posted some code to start/stop a service using Wiseman and WinRM to the Wiseman's user's mailing list on 6/18/2009. The problem with Wiseman is that you need to set up your WinRM to use unencrypted communication and basic authentication. Because the encryption is relatively new and "invented" by MS called "HTTP-SPNEGO-session-encrypted" the RFC (Google cache -- as you might expect from MS:-) is kind-a cryptic...

But we can do something about the authentication and use Negotiate by using the SpnegoHttpUrlConnection implementation from the SPNEGO project. Make sure to read through all their pre-flight documentation to set up kerberos and so on (keyword: krb5.conf, login.conf) The account you use needs to be at least a local administrator and it is helpful to prefix the domain for the user name (MY_DOMAIN\MY-USERNAME) Once this works you can rework Denis' example and have it support the Negotiate protocol as I did:


/**
* Copyright (C) 2006-2009 Hewlett-Packard Development Company, L.P.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
*  http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*
* Authors: Denis Rachal (denis.rachal@hp.com)
*/
package com.hp.wsman.client.transport;

import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.net.Authenticator;
import java.net.HttpURLConnection;
import java.net.PasswordAuthentication;
import java.net.URL;
import java.net.URLConnection;
import java.security.KeyManagementException;
import java.security.NoSuchAlgorithmException;
import java.security.PrivilegedActionException;
import java.security.SecureRandom;
import java.util.Map.Entry;
import java.util.logging.Level;
import java.util.logging.Logger;

import javax.net.ssl.HostnameVerifier;
import javax.net.ssl.HttpsURLConnection;
import javax.net.ssl.SSLContext;
import javax.net.ssl.TrustManager;
import javax.net.ssl.X509TrustManager;
import javax.security.auth.login.LoginException;
import javax.xml.bind.JAXBException;
import javax.xml.soap.SOAPException;
import javax.xml.soap.SOAPMessage;

import org.ietf.jgss.GSSException;

import net.sourceforge.spnego.SpnegoHttpURLConnection;

import sun.misc.BASE64Encoder;

import com.sun.ws.management.Message;
import com.sun.ws.management.addressing.Addressing;
import com.sun.ws.management.transport.ContentType;

public final class HttpClient {

private static final Logger LOG = Logger.getLogger(HttpClient.class
  .getName());
private static PasswordAuthentication auth;

private HttpClient() {
}

static {
    System.setProperty("java.security.krb5.conf", "C:\\Users\\eichbege\\wiseman-client\\client\\src\\krb5.conf");
    System.setProperty("sun.security.krb5.debug", "false");
    System.setProperty("java.security.auth.login.config", "C:\\Users\\eichbege\\wiseman-client\\client\\src\\login.conf");
}

static class MyAuthenticator extends Authenticator {
      public PasswordAuthentication getPasswordAuthentication() {
          // I haven't checked getRequestingScheme() here, since for NTLM
          // and Negotiate, the usrname and password are all the same.
          System.err.println("Feeding username and password for " + getRequestingScheme());
          return auth;
      }
  }


public static void setPasswordAuthentication(
  final PasswordAuthentication pauth) {
 auth = pauth;
}


public static void setTrustManager(final X509TrustManager trustManager)
  throws NoSuchAlgorithmException, KeyManagementException {

 final TrustManager[] tm = { trustManager };
 final SSLContext sslContext = SSLContext.getInstance("SSL");
 sslContext.init(null, tm, new SecureRandom());
 HttpsURLConnection.setDefaultSSLSocketFactory(sslContext
   .getSocketFactory());
}

public static void setHostnameVerifier(final HostnameVerifier hv) {
 HttpsURLConnection.setDefaultHostnameVerifier(hv);
}

private static HttpURLConnection initConnection(final String to,
  final ContentType ct, Object msg) throws IOException {
 if (to == null) {
  throw new IllegalArgumentException("Required Element is missing: "
    + Addressing.TO);
 }

 if (auth != null) {
          //Authenticator.setDefault(new MyAuthenticator());
        
//          String encodedUserPassword = new BASE64Encoder().encode((auth
//                  .getUserName()
//                  + ":" + new String(auth.getPassword())).getBytes());
//          conn.setRequestProperty("Authorization", "Basic "
//                  + encodedUserPassword);
      }

 final URL dest = new URL(to);
 URLConnection conn = null;
 SpnegoHttpURLConnection spnego = null;
 try {
     spnego = new SpnegoHttpURLConnection("custom-client", auth.getUserName(), auth.getPassword().toString());
     spnego.setRequestMethod("POST");
     spnego.setRequestProperty("Content-Type",
                ct == null ? ContentType.DEFAULT_CONTENT_TYPE.toString() : ct
                        .toString());
        spnego.setRequestProperty("User-Agent", "https://wiseman.dev.java.net");
        spnego.setRequestProperty("Accept", ContentType.ACCEPTABLE_CONTENT_TYPES);
        conn = spnego.connect(dest, transfer(msg));
      } catch (GSSException e) {
          // TODO Auto-generated catch block
          e.printStackTrace();
      } catch (PrivilegedActionException e) {
          // TODO Auto-generated catch block
          e.printStackTrace();
      } catch (LoginException e) {
          // TODO Auto-generated catch block
          e.printStackTrace();
      } catch (SOAPException e) {
          // TODO Auto-generated catch block
          e.printStackTrace();
      } catch (JAXBException e) {
          // TODO Auto-generated catch block
          e.printStackTrace();
      }

  //Authenticator.setDefault(new MyAuthenticator());
  //conn = dest.openConnection();

//


 final HttpURLConnection http = (HttpURLConnection) conn;
 //http.setRequestMethod("POST");

 return http;
}

// type of data can be Message or byte[], others will throw
// IllegalArgumentException
private static ByteArrayOutputStream transfer(final Object data)
  throws IOException, SOAPException, JAXBException {
    ByteArrayOutputStream os = null;
 try {
  os = new ByteArrayOutputStream();
  if (data instanceof Message) {
   ((Message) data).writeTo(os);
  } else if (data instanceof SOAPMessage) {
   ((SOAPMessage) data).writeTo(os);
  } else if (data instanceof byte[]) {
   os.write((byte[]) data);
  } else {
   throw new IllegalArgumentException("Type of data not handled: "
     + data.getClass().getName());
  }
  return os;
 } finally {
  if (os != null) {
   os.close();
  }
 }
}

public static Addressing sendRequest(final SOAPMessage msg,
  final String destination) throws IOException, SOAPException,
  JAXBException {

 if (LOG.isLoggable(Level.FINE))
  LOG.fine("\n" + msg + "\n");
 final HttpURLConnection http = initRequest(destination, ContentType
   .createFromEncoding((String) msg
     .getProperty(SOAPMessage.CHARACTER_SET_ENCODING)), msg);
 //transfer(http, msg);
 final Addressing response = readResponse(http);
 if (LOG.isLoggable(Level.FINE)) {
  if (response.getBody().hasFault())
   LOG.fine("\n" + response + "\n");
  else
   LOG.fine("\n" + response + "\n");
 }
 return response;
}

public static Addressing sendRequest(final SOAPMessage msg,
  final String destination, Entry... headers)
  throws IOException, SOAPException, JAXBException {

 if (LOG.isLoggable(Level.FINE))
  LOG.fine("\n" + msg + "\n");
 final HttpURLConnection http = initRequest(destination, ContentType
   .createFromEncoding((String) msg
     .getProperty(SOAPMessage.CHARACTER_SET_ENCODING)), msg);
//  if (headers != null) {
//   for (Entry entry : headers) {
//    http.setRequestProperty(entry.getKey(), entry.getValue());
//   }
//  }
//
//  transfer(http, msg);
 final Addressing response = readResponse(http);
 if (LOG.isLoggable(Level.FINE)) {
  if (response.getBody().hasFault())
   LOG.fine("\n" + response + "\n");
  else
   LOG.fine("\n" + response + "\n");
 }
 return response;
}

public static Addressing sendRequest(final Addressing msg,
  final Entry... headers) throws IOException,
  JAXBException, SOAPException {

 if (LOG.isLoggable(Level.FINE))
  LOG.fine("\n" + msg + "\n");
 final HttpURLConnection http = initRequest(msg.getTo(), msg
   .getContentType(), msg);

//  if (headers != null) {
//   for (Entry entry : headers) {
//    http.setRequestProperty(entry.getKey(), entry.getValue());
//   }
//  }

 //transfer(http, msg);
 final Addressing response = readResponse(http);
 if (LOG.isLoggable(Level.FINE)) {
  if (response.getBody().hasFault())
   LOG.fine("\n" + response + "\n");
  else
   LOG.fine("\n" + response + "\n");
 }
 response.setXmlBinding(msg.getXmlBinding());
 return response;
}

public static HttpURLConnection createHttpConnection(String destination,
  Object data) throws SOAPException, JAXBException, IOException {
 final HttpURLConnection http = initRequest(destination, null, null);

 return http;
}

private static HttpURLConnection initRequest(final String destination,
  final ContentType contentType, Object msg) throws IOException {

 final HttpURLConnection http = initConnection(destination, contentType, msg);
 return http;
}

private static Addressing readResponse(final HttpURLConnection http)
  throws IOException, SOAPException {

 final InputStream is;
 final int response = http.getResponseCode();
 if (response == HttpURLConnection.HTTP_OK) {
  is = http.getInputStream();
 } else if (response == HttpURLConnection.HTTP_BAD_REQUEST
   || response == HttpURLConnection.HTTP_INTERNAL_ERROR) {
  // read the fault from the error stream
  is = http.getErrorStream();
 } else {
  final String detail = http.getResponseMessage();
  throw new IOException(detail == null ? Integer.toString(response)
    : detail);
 }

 final String responseType = http.getContentType();
 final ContentType contentType = ContentType
   .createFromHttpContentType(responseType);
 if (contentType == null || !contentType.isAcceptable()) {
  // dump the first 4k bytes of the response for help in debugging
  if (LOG.isLoggable(Level.INFO)) {
   final byte[] buffer = new byte[4096];
   final int nread = is.read(buffer);
   if (nread > 0) {
    final ByteArrayOutputStream bos = new ByteArrayOutputStream(
      buffer.length);
    bos.write(buffer, 0, nread);
    LOG.info("Response discarded: "
      + new String(bos.toByteArray()));
   }
  }
  throw new IOException(
    "Content-Type of response is not acceptable: "
      + responseType);
 }

 final Addressing addr;
 try {
  addr = new Addressing(is);
 } finally {
  if (is != null) {
   is.close();
  }
 }

 addr.setContentType(contentType);

 return addr;
}

public static int sendResponse(final String to, final byte[] bits,
  final ContentType contentType) throws IOException, SOAPException,
  JAXBException {
 final HttpURLConnection http = initConnection(to, contentType, bits);
 return http.getResponseCode();
}

public static int sendResponse(final Addressing msg) throws IOException,
  SOAPException, JAXBException {
 final HttpURLConnection http = initConnection(msg.getTo(), msg
   .getContentType(), msg);
 return http.getResponseCode();
}
}

Supposedly you could achieve the same with the standard HTTP connection from Java 6 (see tutorial) but I couldn't get to work. So who knows... anyway the SPNEGO project's solution worked for me with an unencrypted setup on Windows Vista.

Spring and REST

2009-12-07T22:04:00.000-08:00

After loving for more than two years Don Brown's fantastic REST plugin for Struts2 I decendent again into Spring's version of REST (see http://blog.springsource.com/2009/03/08/rest-in-spring-3-mvc/) My simple porting of Don Brown's demo app to Spring failed again (I tried 12 months ago to use some work from carbon five) Anyway why, why does it not work I thought I had it figured out:


package edu.ucsd.extension.springmvchelloworld;

import java.util.Collection;

import org.springframework.stereotype.Controller;
import org.springframework.ui.Model;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.servlet.ModelAndView;



@Controller
public class OrdersController {
    
    private Order model = new Order();
    private Collection list;
    private OrdersService ordersService = new OrdersService();

    // GET /orders/1
    @RequestMapping("{orderId}")
    public String show(@PathVariable String id, Model model) {
     System.out.println("Showing order " + id);
  setId(id);
        model.addAttribute("order", getModel());
        return "show";
    }

    // GET /orders
 @RequestMapping
    public String index(Model model) {
        list = ordersService.getAll();
        model.addAttribute("orders", list);
        return "orders-index";
    }
    
    // GET /orders/1/edit
 @RequestMapping("/{orderId}/edit")
    public String edit(@PathVariable String id, Model model) {
  this.setId(id);
  model.addAttribute("order", getModel());
        return "orders-edit";
    }

    // GET /orders/new
 @RequestMapping("/new")
    public String editNew(Model model) {
        model.addAttribute("order", new Order());
        return "orders-editNew";
    }

    // GET /orders/1/deleteConfirm
 @RequestMapping("/{orderId}/deleteConfirm")
    public String deleteConfirm(@PathVariable String id) {
  this.setId(id);
        return "orders-deleteConfirm";
    }

    // DELETE /orders/1
 @RequestMapping(value="/{orderId}", method=RequestMethod.DELETE)
    public String destroy(@PathVariable String id) {
        ordersService.remove(id);
        //addActionMessage("Order removed successfully");
        return "orders-index";
    }

    // POST /orders
 @RequestMapping(method=RequestMethod.POST)
    public ModelAndView create(Order model) {
        ordersService.save(model);
        return new ModelAndView("orders-show", "model", getModel());
    }

    // PUT /orders/1
 @RequestMapping(value="/{orderId}", method=RequestMethod.PUT)
    public String update(@PathVariable String id, Order model) {
        ordersService.save(model);
        return "order-index";
    }

    public void setId(String id) {
        if (id != null) {
            this.model = ordersService.get(id);
        }
    }
    
    public Object getModel() {
        return (list != null ? list : model);
    }

}

Well, I guess there is some other trick I am missing.

BTW: the repository in the POM file needs to look like:


 
  org.springsource.maven.snapshot
   Springframework milestone
   http://maven.springframework.org/milestone

The Passionate Programmer

2009-11-15T18:56:00.000-08:00

I have been reading the The Passionate Programmer: Creating a Remarkable Career in Software Development (Pragmatic Life) and I think it is an absolute must read. I stumbled upon it on Matt's blog (see http://raibledesigns.com/rd/entry/the_passionate_programmer_by_chad) and I immediately liked the book. Chad is doing an incredible job explaining how to stay relevant and increase your career in "computers". The most impressive pieces are when he explains how to function in a corporate environment. "Make your manager happy he controls your pay" I can't say often enough how important that is. He also prefers a role maintaining software over some new project because oddly enough it gives you more freedom. His comparisons with the music scene are refreshing however probably not for everyone...

Anyway for $15 it's a must read :-)

Springing in Action

2009-08-28T20:08:00.000-07:00

I have been asked to develop an online class for UCSD Extension covering Spring -- I am really excited and I will try to get it going soon --

On another note I started a new job with HP Software working among other things with JSF -- soon I can say I have seen all the web frameworks in the world :-)

SDCodeCamp

2009-06-26T10:23:00.001-07:00

Saturday, 6/27 I will be talking on Scala and Spring -- check out their home page (http://www.socalcodecamp.com/)

IntelliJ always errors when loading Scala facet

2009-05-25T20:53:00.000-07:00

The last couple of weeks every time I opened my project with IntelliJ I would get an error with the Scala plugin and the system would become unusable while indexing all the time.

I considered switching from my usual stack Struts 2/Spring/Hibernate to Lift/JPA and Eclipse but after I invalidated the cache (File -> Invalidate Caches) it seems to work again.

Simple cron like scheduler in Scala

2009-03-27T20:23:00.000-07:00

Today in my Scala explorations I ran into the problem that I wanted some scheduler like Quarz or java.util.Timer -- that should be easy in scala I thought an came up with the following:

private val timedActor = actor {
    //once a day
      while (true) {
      performTask
      val c = Calendar.getInstance
      c.set(Calendar.HOUR_OF_DAY, 0) //midnight
        c.set(Calendar.MINUTE,0)
      c.add(Calendar.DAY_OF_YEAR, 1) //next day
        val sleepAmount = c.getTimeInMillis - Calendar.getInstance.getTimeInMillis
      Thread.sleep(sleepAmount)
    }
}

Pretty ~~slick~~ heavy weight (check the comment for a better implementation) -- they should include that in one of their libraries...

HTML Screen Scraping

2009-03-19T05:40:00.000-07:00

For some little side project I found myself screen scraping some HTML sites for information. My first idea was to access the pages with the URL class and then use TagSoup for parsing (see this Blog Entry for an example). This in fact worked quite well and using XPath from Scala was a blast.

Nevertheless the scraping sometimes didn't work because for some weird reason the site I was scraping demanded a JavaScript enabled browser (and sending forms is no real fun with that approach). So I turned to HTMLUnit which seems to be an even better screen scrape tool.

Now what we really need is a HTMLUnit which gives us simple access to a TagSoup of the content...

Java and Concurrency

2009-02-02T09:51:00.000-08:00

Check out Alex Miller's fantastic blog posts at http://www.nofluffjuststuff.com/blog/alex_miller/2009/01/java_concurrency_bugs_3__atomic__atomic__atomic.html

Full program @ UCSD

2009-01-07T12:16:00.000-08:00

Here is my teaching schedule for this quarter:

Introduction to the Spring Framework (Winter 2009)

Java Programming I: Introduction to Java Programming (Winter 2009)

Mapping Objects to Relational Databases using Hibernate (Winter 2009)

Struts 2.0 (Winter 2009)

That's plenty and I am really excited about that line-up --