I spent a big chunk of my day dealing with a project that is, in no uncertain terms, a trainwreck. The client has sunk a ton of money into a product which is in, its current (first phase supposedly finished) state, unusable (client and vendor shall remain unnamed.) My role in the project has been strategic and as a liason, not technical, which to some extent gives me a bit of a distanced view.
Web development trainwrecks are, sadly, far from isolated cases – they happen all the time, even when all of the parties have good intentions. And as someone who is building a business around doing this sort of work, it is of keen interest to me as to why some projects end up in the state that this project is in, and I want to make sure to avoid these kinds of situations. So how do we avoid trainwrecks? Some trainwrecks we can see coming miles away, but yet we are in complete denial about them. Some trainwrecks are like sudden derailments – it’s not at all clear where it comes from. But I think all trainwreck projects have the seed of the wreck somewhere in the history of the project.
The hallmarks of this particular trainwreck were so clear, that in retrospect, they scream out at me:
- Lack of transparency about development process
- Lack of transparency about cost implications of increased scope
- Waterfall development process (well, the vendor said they practiced Agile, but in practice, it’s been waterfall)
As a practitioner of the Agile development process (we use a somewhat modified form of Scrum, in particular,) I’m beginning to really see the value of this kind of process. It makes visible all sorts of things that are often hidden. It seems like the Agile methodology helps in a number of ways:
- Once educated, clients have a window into the development process. They know what small chunks of development are going to happen in a given time interval, and they know what they will get at the end of that time interval
- Things are developed in priority order
- Clients can critique things early
- New functionality becomes a part of the “product backlog” and it is easier to have clarity about what is and is not within scope
Of course, it is theoretically possible to be completely transparent in a traditional waterfall methodology, and completely opaque using Agile, but I do think that the Agile methodology makes it way more difficult to be opaque. But it also takes some work and education of clients unfamiliar with the methodology (as well as making mistakes along the way on our part as developers.)
And I’ve been able to watch this process work well, not only with our own projects, but also with a project I was a strategic lead on. I was pretty skeptical a year or so ago, but now I’m sold. And since transparency has always been something of real importance to me, a development process that encourages transparency is a good thing.
Tagged as:
agile,
Development,
nptech,
web
As you can tell, I haven’t had much time to blog lately. Here are some great links I’ve come across that I thought were worth sharing:
Tagged as:
nptech,
opensource
I don’t have kids, but I do know how young kids ask questions. They are innocent, and free of assumptions, and keep asking “why?” In the end, the poor adults either get tired of the questions, or realize that there are assumptions they’ve been making for all this time that might actually be worth questioning.
Human processes mold around software. We see this all the time. A CRM gives you these 5 canned reports, and you get used to making do with what’s there. A legacy client database requires a certain order of data entry, and your intake forms have been produced to copy that order. Your email software has particular limitations, and you find behavioral workarounds.
What’s also true in the realm of customized software, is that software is molded around people. You put in your RFP that a package spit out data in X,Y and Z ways because your ED is used to data in that form (maybe because a package they had at their previous organization had those canned reports.) You have a requirement that data be entered into the system in one particular way, probably because that’s the way you’ve always done it. Sometimes, you feel the need to replicate a process that the person 3 administrative assistants ago put in place that was molded around their particular limitations, just because that’s what you know.
When you are undergoing the process of creating or implementing a new system of any sort, whether it be a CMS for a website, a CRM, some internal system, it is a really good exercise to be like a 3 year-old, and keep asking “why?” Why do we need this feature? Why will this report be important? Why should the software work this way? Once you peel the layers down to the bottom, you’ll either have “we don’t know” or “because we believe it will help us meet our mission in this specific way.” Then you know what you should take, and what you can leave behind.
Tagged as:
nptech,
Software
I got to spend one day at CiviCRM developer camp this week. Unfortunately, it came after 4 long days of conferencing, after many exhausting days of work, so I wasn’t at my peak. But I learned a lot, and thought I’d share some of what I took away from that day.
First, the core team shared some of the new stuff coming out in version 2.3, and it is awe-some. One of the major reasons CiviCRM gets dinged as a CRM/DMS is that it doesn’t have reports. Well, that problem is about to go away with the release of CiviReport in 2.3. There will be a number of canned reports, and some really nice ways to create reports. Plus charts! Yay! There were some pie charts, and regular bar charts. I don’t have the new svn trunk of CiviCRM installed, otherwise, I’d show some screenshots, but it looked really nice. (I’ll be installing CiviCRM from svn in the next week, and I’ll probably blog more as 2.3 develops.)
There are some really nice usability improvements coming up in 2.3 as well – to make the basic contact pages much easier to navigate. And there is a new menu system, which will make things a lot easier. And, for Drupal users, some sweet Views 2 and CCK integration.
CiviEvent is getting waiting lists, registration approval, and user-modifiable registrations, and some other improvements.
The Alpha of 2.3 should be out by July.
I also learned about CiviCase, which is actually present in 2.2. I saw the example of it used for the Physician Health Program in Canada. It’s quite good, and there are some useful docs to see it at work on the CiviCRM wiki. I’d love to find an organization, such as a small human services organization, in need of case management software, that could use CiviCase – it would be a great, and relatively inexpensive alternative to current offerings out there. And more organizations using CiviCRM for case management would help CiviCase get even better.
I also dug into some of the internals and code of CiviCRM, and feel better equipped to start contributing more than ideas and feedback to the project.
Tagged as:
civicrm,
CRM,
nptech,
opensource
NTC is coming, and I don’t have to pack! That’s a good thing. But I will be BARTing my way into SF everyday, from Saturday for Penguin Day, Sunday through Tuesday for NTC, and Wed and Thursday for CiviCRM Developer Camp. I’m very much looking forward to all of it, even though it seems like it’s going to be an exhausting 6 days.
I’d love to meet new folks and see as many old friends as possible, so I figured I’d share where I’ll be during these days, and perhaps we can meet up. You can email me, @pearbear on twitter, or give me a text message or call … if you know my cell, that is. :-)
Tagged as:
09ntc,
civicrm,
nptech,
nten,
penguinday
In general, the activities of the big tech corporations have somewhat limited and indirect effect on nonprofit technology. For large enterprises, the activities of the big players is a much more immediate and important set of issues to deal with. For us, it’s generally much more removed.
However, today’s news that Oracle is going to buy Sun Microsystems has some very important implications. Why? It has to do with the fact that many, many nonprofit websites and web applications are built using MySQL, the most popular open source database management system. Sun bought MySQL AB (the company behind MySQL) last year for $1 Billion dollars, and therefore, MySQL AB now becomes a part of Oracle, it’s primary competition.
There is some suspicion that there may be anti-trust challenges because of this, but if it goes through, it raises some huge questions about what happens to MySQL because of this. Of course, since MySQL is open source, there is no danger of MySQL going away, someone can always fork it. And, ultimately there is a great open source database alternative called PostgreSQL, but support for it is not universal. However, the future of ongoing support and development for MySQL is certainly in question. Most nonprofits don’t get any support from MySQL AB directly, but larger organizations that might have been getting some support might see changes down the road.
It’s something that those of us who depend on MySQL for our web development projects will be watching quite closely.
Tagged as:
database,
mysql,
nptech,
oracle,
sun
On the right is my blogroll, that needs updating, but I thought I’d do some shout outs to blogs I’ve lately been loving and really learning a lot from, who are probably not on that list (yet).
- Wireframes Magazine – I’ve been doing Information Architecture for a very long time, now, but it’s great to learn new tricks and tools.
- Flowing Data – OK, I’ll fess up, I’m a data geek. And I love data visualizations, and ways to make data easily accessible. I am so envious of people with graphics skills who can do that well. There are a whole lot of really cool things here.
- RoughType by Nicholas Carr – really smart dude, really interesting stuff.
- ONE/Blog – ONE/Northwest never ceases to amaze me
- The Open Road – Matt Asay has some interesting insights from the Open Source biz world
Tagged as:
nptech
Third to last in my series on CMS and CRM integration (next up, Joomla and Salesforce, followed by Drupal and Salesforce) is using web forms.
I wanted to talk about this because it is arguably the most common form of “integration” between CRM and CMS that’s out there (besides the manual kind). You’ve got a CMS, and you’ve got a CRM somewhere else, and you need some way for data from users to make it to your CRM. Of course, it’s not really integration – there is no sharing of data between the CMS and the CRM in any useful way. But webforms can really help you get things done. Here are some examples of things I’ve done and seen done:
- A custom donation page that’s sitting on a service like Network for Good that is linked from the website, or framed within it
- The HTML for a “Web to Lead” form from Salesforce.com pasted into a CMS page
- The HTML for a event registration form or donation form that goes to a hosted service
In the first option, the form isn’t hosted at all on your site. In this option you have the least control over look and feel – the vendor controls the look and the behavior. An example of this I’ve run into is when an organization uses Blackbaud’s Raiser’s Edge, and wants to have online donations via NetSolutions, their older (and much cheaper) “integration” tool. They provide a page, which hooks directly into the users RE installation. But you can’t customize the page in any useful way, so if you’ve just designed a brand-spanking new site, this page is gonna look like crap. (Luckily, at least Network For Good’s donation pages look snappy and nice, but are going to look a lot different than your website.)
The other options are much better for look and feel – you can take the HTML, and, in most instances, style it to look like your site. You can even sometimes include Javascript for validation or other functionality. But this is still strictly one-way communication – the form data goes directly to the service (and does not pass go.) You don’t get any of it.
This is a great start to integration, if your budget doesn’t allow for true, deep, two-way integration between CRM and CMS. And it’s a great way to get your feet wet in thinking about what you might want to do with CRM and CMS. And, in some instances, depending on both CRM and CMS, it might be your only option.
Tagged as:
CMS,
CRM,
integration,
nptech
I love Penguin Day. One of my favorite days of the year. Always comes right around NTC. This year, it’s before NTC, on Saturday, April 25. It’s a day dedicated to conversation and community around nonprofits and open source software. There’s some great stuff on the Agenda, like:
- Introduction to Free and Open Source Software
- Fundraising with all free software
- Free And Open Source Online Advocacy: Tools And Best Practices
- Making sense of Free and Open Source Content Management Systems
- Introduction to Blogging with Wordpress
- Intro and Advanced sessions on Joomla! and Drupal
- CiviCRM vs Salesforce.com: What Are the Differences?
- Mobile Volunteering: The ExtraOrdinaries Project
- Creative Commons And Open Content
- And many more…
You can register at Penguinday.org. Thanks to the generosity of Google, we’re delighted to grant fee waivers to anyone who needs one!
I look forward to seeing folks there.
Tagged as:
nptech,
ntc09,
opensource,
penguinday09
Now that the Idealware CMS report is out, I get to have my say about it. Here’s the first post, there might be more to come.
The thing that is prompting this post is the little storm about the security metric that we used to try and get a handle on the security of the 4 different systems we reviewed. More on that in a bit.
You might think that comparing four different open source packages that, in essence, do pretty the same thing (in a broad sense) would be a cakc walk. In fact, nothing could be farther from the truth. The developers of each project have completely different sets of assumptions about what the right way to do things is, and completely different philosophies and ethos when it comes to building interfaces and functionality. Making apples-to-apples comparisons of these systems was one of the most difficult analytical tasks I’ve taken on in a while (and, actually much of the heavy lifting of designing the analysis was done by Laura Quinn), and until you attempt such a thing, please be somewhat tempered in your complaints about it.
Now the security issue. One of the 12 different aspects we are comparing is “Scalability and security”. The report isn’t about security, it’s a very, very broad comparison of the systems, with security as a very small component. That’s just the context. Two (yes, just two) questions out of many relate to security. First, a simple metric relating to security reports, and second, what processes are in place in the communities to deal with security. This wasn’t designed to be an in-dept, complex analysis of security. If it had been, we would have done a lot more work on how to measure security. On the Four Kitchens blog, they say, “While both reports above seem to identify Drupal (and Joomla! and WordPress, to be fair) as having notably bad [emphasis mine] security, they’re also both based on one superficial metric: self-reported vulnerabilities.” Now I can’t speak about the IBM report (I haven’t even read it yet), but our report says no such thing. Drupal gets a “Solid” on Scalability and security. Solid, which is only one step below Excellent. And you know why it got a “Solid”? Because, indeed, it does have more reported security vulnerabilities than Plone (as do Joomla and WordPress.)
David Geilhufe, who also takes issue with the security metric, has some good points. Yes, sheer numbers of vulnerabilities are not anywhere near the best metric of whether or not a system is secure or not. As a quick comparative look between a small number of open source systems, it’s hard to argue that it contributes no information. Four Kitchens seems to suggest that part of the reason for more vulnerabilities in Drupal compared to Plone is that it’s more popular. But, if you’ve been an observer to the Linux/Windows FUD wars, you’ll remember that Microsoft has that exact same argument about why there are more security vulnerabilities in Windows as compared to Linux. And the Linux folks say, in response, “It’s not popularity, it’s design.” I’m sure that Four Kitchens, and most open source software developers agree with that perspective. In reviewing Plone, and talking with people who develop for Plone, I was convinced that the reason that Plone had fewer reported vulnerabilities was not just because it was less popular – it’s because it (and Python and Zope) was more secure by design.
I am completely happy with Drupal’s security (otherwise, it wouldn’t have gotten a “Solid.”) I think the Drupal community takes security extremely seriously, and if they didn’t, I wouldn’t have chosen it as a platform for development. I also think that the Joomla and WordPress communities take security seriously. In our estimation, they were all really good. But Plone was just that much better.
Tagged as:
CMS,
drupal,
idealware,
plone,
security