Programmatic Paralysis Or a senior developer’s hobby problem

Over the course of my career, heck, over the course of my life, I’ve had a reoccurring, “issue” shall we say. In a nutshell, I feel like doing some programming, but what shall I write?

You see, programming for me, since the age of nine, has been my hobby. I just happen to be one of those guys that actually gets paid to do what he loves to do, and would be doing anyway. For the time being, I will ignore the downsides of that as it’s a post and then some in it’s own right Winking smile 

What got me into it was really two things. The first is learning things. That “Eureka” moment when moments ago a bunch of mumbo-jumbo clicks in and make sense is an awesome feeling. The other draw was making some machine do my bidding. Again an awesome feeling which I can’t put into words if you’re not a programmer, I’m sure there are analogs in other disciplines, however I’m not familiar with those disciplines.

Well that is all fine and good, but I’ve been a professional (e.g. my sole income) programmer for seventeen years. I’ve got a lot of experience at a lot of levels of business types (differing in both type of industry and company size. Size may not be the right phrase, I’m trying to speak towards the level of “enterprise” processes an organization may or may not embrace)

Every now and again I get lucky and a new relatively self contained technology emerges that I’d like to toy around appear (I’m thinking Linq, jQuery, WebService, etc) Things that are big enough to have a challenge but small enough to not require a massive underlying framework.

The latter is really the problem, if I’m going to spend any of my not very often free time to writing code, I need to be getting something out of it. I need it to genuinely teach me something and therefore enhance my skill-set, and thus my career. Or I need it to be fun enough that it’s worth the cost.

An example of fun to be worth the cost, I will cite my work on the Quake3 Arena III mod, ReactanceUnlagged. (sorry the site is really nothing more then where to grab the latest version which is a few years old.) But working on this was a blast! At one time we had a number of people running the mod on their servers and a fair amount of the very waning quake3 community (it was a 10 year old game when I started modding.) It was written in plain jane C which is my favorite play language, mostly because while it’s possible to blow off your foot, it’s really rewarding when you don’t and it works right Open-mouthed smile

Now the down side to that project was the ramp up, learning their API’s and their virtual machine you coded to.

So the other type of project I’d be drawn to for learning experiences is something more enterprise level, where I’s need to design and implement a database, a data access layer to access the data, and a back-end to process the data, not to mention the ui to work with the data (I’m not really a UI guy, but I can make something clean and jQuery is fun) So that’s a lot of ramp up to. A lot of the work I already know how to do and to do well, but implementing it again just to have something to work off to learn something is rough.

The same applies to open source projects, the ramp up is rough, while the rest is rewarding.

I seem to be having a problem picking where I want to invest the ramp up time, and while I believe in supporting and working on open source projects, I also feel that if I have something unique to contribute to the world (even if it’s just my own immediate world) I’d rather do that.

And the more time passes as I can’t decide, the more time I’m not ramping up and wasting.  Most likely I should just pick something and go, worst case I find something else that rips me away which must mean I was more interested in it than the other, so be it.

jQuery Autocomplete against an MVC driven JSON data provider

First, a disclaimer, this is basically some prototyping I did to discover how feasible this stuff is in reality. It is not an attempt to provide any sort of framework for consumption of others, but arguably the concepts could be used to that end. Feel free to appropriate any of it at will in any way you see fit. All shout-outs accepted 😀

For the sake of having some context around what I am doing, I have a Data Layer provided by the Microsoft Entity Framework running against an existing database. My MVC application only currently deals with a Client model which is related to the Clients table in my database (I take no responsibility for database naming conventions here, I worked with what I had.) I have a Client Controller, a Client Model (very simplified First, and Last Names; Id, and ClientCode properties relating to columns of similar names in the database), and various Client Views (not surprisingly, an Index, Detail, Edit, Create, and Delete views.)

With the above in place I decided to make a simple Client Picker that would work by the user typing the Client’s Last Name into a textbox and it would autocomplete and narrow the choices as the user typed. The final picked Client should be rendered in the Textbox as “Lastname, Firstname, [ClientCode]”

Well first things first, below is the code for the JsonMarshaller class which wraps calling DataContractJsonSerializer. Basically, you pass it a class that is adorned with a DataContractAttribute and it spits out the contents of the class as a Json encoded string.

using System;
using System.IO;
using System.Runtime.Serialization;
using System.Runtime.Serialization.Json;
using System.Text;
using System.Xml;

namespace MvcSandbox.Library.Helpers
{
public class JsonMarshaller
{
public static string CreateJsonString(object objectToSerialize)
{
var dataContractAttribute = Attribute.GetCustomAttribute(objectToSerialize.GetType(), typeof(DataContractAttribute));

var output = new StringBuilder();
var json = new DataContractJsonSerializer(objectToSerialize.GetType());
var ms = new MemoryStream();
XmlDictionaryWriter writer = JsonReaderWriterFactory.CreateJsonWriter(ms);

json.WriteObject(ms, objectToSerialize);
writer.Flush();

byte[] buffer = ms.GetBuffer();
var bytes = new byte[ms.Length];
for (long l = 0; l < ms.Length; l++)
{
bytes[l] = buffer[l];
}

return Encoding.Default.GetString(bytes).Trim();
}
}
}

At this point let’s wire the JsonMarshaller into an MVC web application. For the sake of argument, I created a Controller to handle this. There is no reason it could not have been handled by the existing Client controller, and to be honest this is where I feel it should be for a consistent URL scheme. But when I started I wanted something slightly different from the standard Client views this was a case of me fighting what MVC was trying to give me by default, structure based on what it did (working with a client) then how it did it (being a data service).

To illustrate, to view a client detail the url would be
http://localhost/Clients/Detail/45

and I wanted to get a list of clients with last names starting with “Min” to look like:
http://localhost/Services/Clients/min

In the end I would have rather it be:
http://localhost/Clients/min

In it’s current state I ended up with:
http://localhost/Services/GetClientsByLastName/min

Routing semantics, do what’s best for your app, I just wanted to explain what I’ve got going on here to avoid confusion as MVC is a big believer in naming by convention.

Below are the Client Model and the Services Controller:

Client Model

using System.Collections.Generic;
using System.Linq;
using System.Runtime.Serialization;
using MvcSandbox.Library.DataAccess;
using MvcSandbox.Library.Model;
using MvcSandbox.Library.Repositories;

namespace MvcSandbox.Models
{
[DataContract]
public class ClientModel : ModelBase
{
[DataMember]
public int ClientId { get; set; }

[DataMember]
public string FirstName { get; set; }

[DataMember]
public string LastName { get; set; }

[DataMember]
public string ClientCode { get; set; }

public ClientModel() { }

public ClientModel(int clientId)
{
LoadFromDataEntity((Clients)Repository.Get(clientId));
}

public ClientModel(Clients client)
{
LoadFromDataEntity(client);
}

public static List GetAllClients()
{
return (from Clients client in new ClientRepository().FindAll(20)
select new ClientModel
{
ClientCode = client.ClientCode,
ClientId = client.ClientID,
FirstName = client.FirstName,
LastName = client.LastName
}).ToList();
}

private void LoadFromDataEntity(Clients client)
{
DataEntity = client;

if (client == null)
{
ClearModelFields();
return;
}

ClientId = DataEntity.ClientID;
FirstName = DataEntity.FirstName;
LastName = DataEntity.LastName;
ClientCode = DataEntity.ClientCode;
}

public override void UpdateDataEntityFromModel()
{
DataEntity.ClientID = ClientId;
DataEntity.FirstName = FirstName;
DataEntity.LastName = LastName;
DataEntity.ClientCode = ClientCode;
}

public override void ClearModelFields()
{
FirstName = string.Empty;
LastName = string.Empty;
ClientCode = string.Empty;
ClientId = 0;
}
}
}

The Client Model Base Class

using System;
using System.Data.Objects.DataClasses;
using System.Runtime.Serialization;
using MvcSandbox.Library.Helpers;
using MvcSandbox.Library.Repositories;

namespace MvcSandbox.Library.Model
{
[DataContract]
public abstract class ModelBase : IModel
where T : EntityObject
where TU : RepositoryBase
{
protected TU Repository = Activator.CreateInstance(typeof(TU)) as TU;
protected T DataEntity;

#region Abstract Members

public abstract void ClearModelFields();

public abstract void UpdateDataEntityFromModel();

#endregion

#region Virtual Members

public virtual void Save()
{
UpdateDataEntityFromModel();
Repository.Save();
}

public virtual string ToJson()
{
return JsonMarshaller.CreateJsonString(this);
}

#endregion
}
}

Services Controller

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Web;
using System.Web.Mvc;
using System.Web.UI.WebControls;
using MvcSandbox.Library.DataAccess;
using MvcSandbox.Library.Helpers;
using MvcSandbox.Library.Model;
using MvcSandbox.Models;

namespace MvcSandbox.Controllers
{
public class ServicesController : Controller
{
public ActionResult GetClientsByLastName(string id)
{
var clientLists = new ModelList();

if (String.IsNullOrEmpty(id))
{
ViewBag.Output = clientLists;
}
else
{
string lastName = id;

var entities = new it01_FSC1Entities();

var cls = (from client in entities.Clients
where client.LastName.StartsWith(lastName.Trim())
orderby client.LastName
select new ClientModel()
{
ClientCode = client.ClientCode,
ClientId = client.ClientID,
FirstName = client.FirstName,
LastName = client.LastName
}).Take(20);

clientLists.Items.AddRange(cls);
}

Response.ContentType = "application/json";
Response.ContentEncoding = Encoding.UTF8;

ViewBag.Output = JsonMarshaller.CreateJsonString(clientLists);
return View();
}
}
}

Nothing really magical there. I would like to go on record stating that calling the data access layer directly in my Controller was an ugly time-saver, I haven’t moved it over to my ClientRepository where it belongs. The Repository insulates my front end / model code from being tied to the database structure. It uses the Entity Framework classes to populate models for me. The Model Base class illustrates some underpinnings of it which aren’t really relevant to the point of this post.

So basically, the Controller grabs the data, populates a class with a DataContract so I can use the JsonMarshaller to Json encode it. Of note in the controller is where it set’s the Response’s content-type to “application/json” and content-encoding to UTF-8. jQuery love’s it some UTF-8.

Once we have the results Json encoded, it sets an item in the ViewBag for use by the GetClientsByLastName View to pump the Json out to the client.

@{
Layout = null;
}
@Html.Raw(ViewBag.Output)

Heh, so little code on that last piece ‘eh 😀

At this point we are ready to consume the HTTP GET enabled data provider with a ClientSearch partial view. Which, is essentially a blue bar with a autocomplete enabled textbox on it to search for a client by last name.

This was the difficult piece. In retrospect, not because it was difficult but because with jQuery, if you’re not writing it everyday, you are forgetting how to write it everyday. And it’s been a while since I’ve been living the jQuery life, however, like a bicycle, you just get back on and pour through docs and examples until it sticks again.

Without further ado, the ClientSearch partial view:

<script language="javascript" type="text/javascript">
var clientSourceArray = [];

function GetClientData(searchTerm) {
var url = "http://localhost:1909/Services/GetClientsByLastName/" + searchTerm;
clientSourceArray = [];

var jqxhr = $.ajax({
url: url,
type: 'GET',
dataType: 'json',
async: false,
success: function (data, status, xhr) {
$.each(data, function (i, clients) {
$.each(clients, function (i, client) {
var clientValue = client.LastName + ', ' + client.FirstName + ' [' + client.ClientCode + ']';
clientSourceArray.push(clientValue);
});
});
}
});

return clientSourceArray;
}

$(document).ready(function () {

//set up the autocomplete form
$("#ClientLastName").autocomplete({
source: function (searchTerm, callback) { callback(GetClientData(searchTerm.term)); },
delay: 100,
minLength: 2
});

//set up the watermarks
var watermark = "Start Typing a Client's Lastname...";
var inputElement = $("#ClientLastName");
if (inputElement.val() == "") {
inputElement.css("color", "#555555");
inputElement.css("font-style", "italic");
inputElement.val(watermark);
}
inputElement.focus(function () {
if (this.value == watermark) {
$(this).css("color", "");
$(this).css("font-style", "");
this.value = "";
}
}).blur(function () {
if (this.value == "") {
this.value = watermark;
$(this).css("color", "#555555");
$(this).css("font-style", "italic");
}
});
});
</script>

<div class="client-search-bar">@Html.TextBox("ClientLastName", "", new Dictionary() { {"class", "client-last-name-input"} })</div>

So now a few comments about that, first, I’m looking for a code highlighter that works better and easier then the one I have…

Well the actual “html” portion is pretty easy, 1 div that uses the MVC HTML helper class to generate a textbox. I’ll also ignore the Watermarking code because it’s unrelated short of making a nicer looking textbox (arguably.)

The bulk of the work is done by the GetClientData() function, it makes an asynchrounous AJAX call to get the Json encoded data for the autocomplete control and the setting up of the actual autocomplete. Which in the end being pretty trivial as it can take a function that takes a searchTerm and a “Add an item to the list to be searched” callback function. The way it is written here, it works but it could be more succinctly done had I truly been in the jQuery zen place. As demonstrated on http://net.tutsplus.com/tutorials/javascript-ajax/how-to-use-the-jquery-ui-autocomplete-widget/ in particular, Step 4: “Attaching the Autocomplete”

//attach autocomplete
$("#to").autocomplete({

//define callback to format results
source: function(req, add){

//pass request to server
$.getJSON("friends.php?callback=?", req, function(data) {

//create array for response objects
var suggestions = [];

//process response
$.each(data, function(i, val){
suggestions.push(val.name);
});

//pass array to callback
add(suggestions);
});
}

Yeah, well I’ll clean that up later. In the end it wasn’t that bad aside from my jQuery rustiness.

Hope this helps you, I know the journey has helped me!

DataContractJsonSerializer / Linq oddity? Nope, it’s just me :-(

[Once again the code is on my other machine but…]

So using Microsoft’s Entity Framework with .NET 4 as a data access layer, I select some entity Client objects properties that meet my criteria and take 20 via linq. Something to the effect of:

var clientList = (rom client in entities.Clients where
client.LastName.StartsWith(“min”) select new ClientInfo() { LastName = client.LastName, FirstName = client.FirstName}).Take(20)

So far so good, nothing magical here. I then assign add it as a range to a new List<ClientInfo>

var clientInfos = new List<ClientInfo>();
clientInfos.AddRange(clientList);

When the DataContractJsonSerializer hits that list, it correctly does it, however it manages to add in quite a few characters (that’s a null character) into the string which prevents it from properly being parsed by the built in jQuery JSON functions.

Scratch that, I’m likely reading an entire memory stream in instead of just the used portion.

Well, in the end, it’s a good idea to double check your thought process lest you be missing the real problem. I had hacked around my initial noticing of the ’s by replacing them out of the string so I could move on and come back later and figure out what was wrong. Done and done. I’ll verify it of course but I bet you a dollar that’s it. Funny how things come to you out of the blue, like when you’re trying to complain about an oddity. Chances are, the oddity is you Open-mouthed smile

Json responses in a MVC web application

I am sure this is obvious to every web developer using Microsoft’s MVC implementation with asp.net, but for those looking to move from standard asp.net to an MVC platform, this might not come to mind.

The scenario is, you want to provide application data to a browser-side UI (for use with auto-complete textboxes, asynchronously updating UI controls (no server-side post-backs, things you use jQuery or AJAX for.)

You’ve got a few options:

  • ASP.NET Web Service
  • ASP.NET Web Page (.aspx)  that returns raw data with an appropriate content-type
  • ASP.NET HTTP Handler (.ashx) that returns raw data with an appropriate content-type
  • WCF Web Service hosted under IIS
  • in fact there are a ton of custom solutions you could use as well, I’m going to ignore them and stick with the Microsoft family of solutions.

First, I’m going to throw out the ASP.NET Web Page option. Sure you can do it but it’s exactly the same the HTTP Handler but more efficient and only require the “code-behind” portion of a page, not the .aspx portion as you are returning raw data in whatever format (Json, Xml, csv, whatever.)

Now you have to consider your system interaction a bit. If you have multiple applications consuming the data, WCF is most likely the way to go. It supports multiple protocols (http, tcp/ip, msmq, etc.) If you have a native client and a web application consuming the data. It’s the way to go.

A standard ASP.NET Web Service would also be usable by both but would only support HTTP, but since you can get that functionality with a WCF service, it’s the way things are moving in Microsoft Land. Having said that, configuration of the services, behaviors, end-points is a pain in the @$$! From experience I can say it was easy to use the proxy classes generated by the service proxy but I am having a very hard time getting it to accept HTTP GET requests. If this was any of the other of the options, it would have been a no brainer (due to their limitations.) Complex problems take complex configuration solutions, nature of the beast.

It’s worth mentioning that native client’s can also consume http web services relatively easily, so if you have a lot of knowledge investment in standard web services and you only need to support web based clients then this isn’t a wrong choice at all.

If you find yourself in a situation where only your web application is going to be consuming your application data you can fall through to the simplest form. Make a Page or Handler in your application and directly serve it. Tapping in to get or post variables, cookies, session, it’s all there at your fingertips. It’s my favorite when I only have to worry about feeding the UI due to it’s ease of use.

Enter the land of MVC

First let’s just get it over with, you absolutely can use a .ashx page in a MVC web application on ASP.NET. You will need to add a new route to send it traffic but after that it works the same.

But since that flies in the face of MVC’s “do it this way, you’ll be happier in the end” (as some like to call “Best Practices” Smile) How can you do it the way MVC wants you to.

Simply as it turns out!

Well, first you make a new Controller and give it an Action. For my case I made a DataServiceController with a Action of “GetClientByLastName”. That does the work of fetching the data.

My model is a string containing a List of Client POCO classes which has been serialized to Json (through the magic of .NET 4.0’s inclusion of DataContractJsonSerializer.) so I don’t need a custom model.

Then make a view with no format and not strongly typed and simply @HTML.Raw() write it out. You’ll also want to set the page response’s content-type to reflect what you’re be outputting. I did this in the Controller, I’m sure there are more way to do it.

Anyway, in the end, using easily coded, standard MVC stuff, I am able to get a Json fromatted list of client’s to work with from my UI simply by fetching from:

[warning, this link won’t resolve] http://mydomin.com/clients/min

And I get a list of at most 20 client’s whose last name begin with “min”. Slick enough for me. Anyway hope it helps, I’ll update this when I have some code samples but the concept is easy. I’m embarrassed it took me so long to put two and two together.

Job decisions -or- Problems I never have

So the past couple of weeks have been interesting for me regarding career options. Perhaps that’s not the right phrase, my career is the same (Software Development, or as we real mean in the trenches say, Programming), the difference is for whom I will be shilling (and I mean that in only the loving-est of ways Open-mouthed smile )

But first, a little background. For the past 3 years I have been a contract developer (with the exception of a six moth stint at a marketing company that was a bad fit for me.) Some of that time I did some Independent contracting, but the rest is W2 work from various consulting companies in the vein of temp work (e.g. I was not an in house employee, there was no bench, I was on site at a client for them or I was in the unemployment line looking for the next opportunity.)

It’s not a bad life, however it also isn’t awesome. It used to be, most jobs in my field were full time permanent employee jobs and contractors were called in for special skill sets or to help out with important things that were time sensitive. You might only work eight out of twelve months, but you would charge a premium rate to hold you over when the contract ran out. With the economy as it is these days (years?) many of the permanent positions were removed and replaced with contractors. Of course this flooded the contracting scene and it’s hard to ask for $100 / hour when there are 20 other guys who want to eat and will do it for $30.

Don’t get me wrong, there are specialty contracting gigs that pay very very well, however you need to be a specialist in whatever sector that is. Specialization can improve your bottom line, but can also remove a lot of options because you only do one thing, albeit you do it very well.

I am what I consider to be a Programming “Generalist”. I know a lot of programming languages and technology and can get the job done across the board. Having said that however, I am not the guy you call when you want a security system for million concurrent users architected (although I could implement it given a designed system Winking smile)

At any rate, very recently I received not one, not two, but three different offers from three different employers, and get this, all on the same day. I have never been in this position before. Hell, I have never been in a position with two offers in the same general time period that would have been close enough for me to actually pick one over the other. I have been very fortunate in being able to keep my unemployed time low, but never have I had to choose. I tend to work a job until I’m fed up and quit or downsized.

Well at first I was quite thrilled, it’s nice to have options. So I did what any analytical person does. I fired up excel and ran the numbers. So far so good, there is a clear winner.

Then came the hard part. The non-financial pros and cons lists. The creation of the lists is not really that hard. The difficulty is really having to weigh and prioritize what is important to your happiness and well being and then deciding the risk value of having to take up a con or give up a pro.

For example, maybe you need time flexibility to be able to get your kids where they need to be, maybe their mother could pick up the slack and it’s a non-issue. Maybe you like a longer commute to allow you to decompress after a day of working, maybe you would rather work from home.

Sometimes when comparing two pros and cons list, some of these items cancel each other out. When they don’t you have to decide what makes it worth while to you to overcome the risk of not having it.

And finally, the hardest part of all, actually picking one. I guess that I had come to rely on not having to make a choice and accepting whatever came my way because I have responsibilities I need to honor, and obligations to keep and it is easier accept what you need when you need it then it is to blame yourself for making a bad choice.

In the end, making a bad choice is fine, it happens, it is how we learn. Don’t be afraid of it, do the math, make the lists, make a call and move on. Maybe you’ll need to change course at some point down the line, but that is life. Rest easy knowing you’ve done your due diligence and did the best you could. The rest is the adventure!

I think my friend George hits the nail on the head with this blog post OODA Loop