December 17, 2004

User Developer Relations: What are The Social Norms

So yesterday's post is still raging down there in comments. I've been thinking about the larger issues, beyond my own loss of data on NetNewsWire. I got very upset about the loss of my data and posted a rant that was too harsh. But I would say that this is what a lot of users feel like when they run into system or software failure, especially if what the system lost is very important to them for some reason. I also realize that developers work very hard to structure and code their systems, and I respect that.

But there is something in-between, that is important to address. There are interface and translation between user and developer... and often these are given short shrift by developers, who appear to users to have all the control in making the software. Users feel that instead of starting user needs assessment and testing well before they begin to write the code, and doing iterative testing through the development process, developers often slap the interface on at the end of development, after deciding how to meet user's goals without actually talking with users, and done little or no user testing. For users, it often feels like the system was designed with only engineer's in mind.

There is also the issue of what the contract is when software is paid for... users often feel that paying for software or services means it should work, regardless of what the stated status of the software is. Developers often feel that their work is laborious, that users aren't appreciative or getting what is intended by the software, and that users should take more responsibility for backups and failures.

I want to know what people believe the social norms are for this new kind of software, that collects or enables social, multidirectional data, creations or communications. Not what the law says, and not what the licensing agreement says, but rather, what are the social norms and expectations each side has? How should the community understand these transactions and sales, and can we agree on some community norms for the sale of services and softwares? Or is this simply too much and therefore will the conflict continue unresolved? These questions of responsibility are not posed with money in mind, but rather, a social responsibility toward each other to either behave independently or interact in a certain way.

The discussion that has come up in the previous post on NetNewsWire is interesting. About 75% of respondents feel that users who pay for software have no recourse, are to blame for the failure, and because of their hostility toward the expression of the complaint, appear to feel that users shouldn't even complain publicly (or on blogs) about purchased software that malfunctions.

Regarding my specific case, I think it's likely that others have gotten software from parties or places other than the website selling the software and then paid for it ... and I think this will continue. People who develop things like it when their stuff spreads virally... and if that viralness takes the form of person-to-person sharing of demos, what is the responsibility of a software seller, who is just selling a license to someone, without the benefit of that person seeing the site, to users in order to notify them of known bugs?

Is a developer selling software responsible for having a Help section, or a bug reporting system (in NetNewsWire's case, their product area refers to a Help section, but 'Help' is not hyperlinked where it is described in the middle of this page, and a Help section is not available anywhere else. Apparently, they have bug tracking for users, but the link is not on this product page, though a commenter on the previous post listed it so it apparently does exist. However, how would users know how to find NNW-bugs)?

Are developers who sell software or services responsible for distinguishing between what they mean with 'Beta' and full release software specifically, and what should users expect from developers who put out Beta software for a charge? What if that Beta purchase is the only way to get the software after the trial period? Do users have some responsibility to developers, whether or not they pay for software? Should they not tell anyone but the developer if they experience a malfunction or warn others to avoid the same circumstances that led to the problem? What if they tell the developer and then experience more problems... do they have a responsibility to other users to warn them of the problems?

Should developers provide a way in the software to backup user data? If a user backs up data independently, have they fulfilled their data responsibilities? Or should users make several backup copies? Where should these copies be, and what is a reasonable expectation of user's to backup? What does this look like?

Should developers be emailing paying customers with software updates that fix known bugs (NNW has an update to the version I have, but I only found out about it due checking their site for help after the second loss of data; they didn't send it, but should they, or should users be expected to visit periodically to see if new versions have appeared)?

Should users expect to be told by developers about the state of the software directly? Where does this notice reside? Should online sellers of software include this information with the sale receipt, the licensing info, or elsewhere? Are users just consumers, or are they partners in the market conversation?

Can we make a constructive list of best practices for both sides? Please comment, and I will list them, along with the comments from the previous post, in the next post.

Posted by Mary Hodder at December 17, 2004 05:45 PM | TrackBack
Comments

Mary - you claim to want to understand the social norms around "this new kind of software," but when people try and explain to you what those norms are, you insist on covering your ears, shouting "blah, blah, blah!" and persisting in your current beliefs. For example, you say:

"The discussion that has come up in the previous post on NetNewsWire is interesting. About 75% of respondents feel that users who pay for software have no recourse, are to blame for the failure, and because of their hostility toward the expression of the complaint, appear to feel that users shouldn't even complain publicly (or on blogs) about purchased software that malfunctions."

Actually, about 75% of respondents feel that *you* are to blame for the repercussions of the failure, because you insisted on using beta software for work you claim was mission-critical to you (this, btw, is what is meant by a "production" environment, a term you indicate in another post on another site that you don't understand).

It isn't that people feel that users shouldn't complain about software that malfunctions, or that these sorts of comments shouldn't be made in public on a weblog - provided you are talking about release quality, not beta software! It doesn't matter if you paid for the beta! It's a beta! By using the beta, you are agreeing to *test* the product for the developer, to help him find bugs so he can fix them. Obviously, you completely fail to understand what beta software is all about!

In the case of the NNW beta, there is in fact quite an elaborate and well established beta testing procedure, through which a large number of users have been testing the software for quite some time. A few weeks ago, the software reached the point where it was released as a "public" beta, meaning you could actually go and download it, without being part of the testing team. Doing a little investigating on Brent's weblog or the Ranchero website would have revealed this information in detail, along with information on how to find the bug database and other useful testing related information.

More importantly, a little reading would have revealed that:

1. NNW 2 was in a public beta phase, had bugs, and would have periodic releases made fixing those bugs until a release candidate was reached.

2. NNW 1.08, an actual production quality piece of software, was available for download - and if you paid for a license for that, you got NNW 2 *for free* when it was released!

3. Alternately, if you wanted to go straight ahead and pay for NNW 2, even though it was still in beta, you could do that. You didn't *have* to though - NNW 1.08 is the stable, production version of the software, the one recommended for people who aren't testing to use - and the license for NNW 1.08 is valid for NNW 2 - so you could buy a NNW 1.08 license, also download the NNW 2 betas, run 1.08 for your mission critical stuff, and play with NNW 2 to see the "fancy" new stuff.

Beyond the specific case of NNW, however, it's evident that you simply don't comprehend beta software and its purpose. You also very clearly don't really understand safe computing strategies like how to implement a good backup strategy.

Yes, it is the developer's responsibility to provide software that functions within the scope of the intended parameters for that software - once that software has been designated as "final" and released to the public as such. No such responsibility *can* exist for software that is still in the testing and development cycle - nor is it a developer's responsibility to back up all of a user's personal data, since there's no way for a developer to anticipate what backup devices or media may be available on a given user's computer (and no, creating another copy of the data on the same hard disk does not constitute a "backup.")

As for making users aware of bugs and updates, again, there is a great deal of difference between beta and release software. Again, a little investigation on your part would have revealed that there is a mailing list for NNW beta testers, through which testers receive regular bug reports and information on updates. Those participating in beta testing are *expected* to participate in this list, both so they know what is going on, and so they can report the bugs they themselves find to the list in general and the developer specifically.

So, rather than castigating Brent and Ranchero for your lost data, you should instead be apologizing - not just for your inexcusable post of yesterday, but because *you were a bad tester*! You failed to carry out your duties as a tester, you failed to take suffcient safeguards to protect yourself from the consequences of using software clearly labeled as unstable, and most damning, you failed to educate yourself about what you were getting involved with.

You were given a whole list of "best practices" in the comments for the previous post - you just didn't want to hear them.

Posted by: Doug Miller at December 17, 2004 08:43 PM

Thanks, Mary -- lots of great questions and ideas. I can't speak to everything, but I can say a few things about NetNewsWire and about what we do.

1. NetNewsWire's Help section is in the Help menu of the application. The first item in the menu opens a NetNewsWire manual in Apple's Help Viewer application.

2. You asked about finding NetNewsWire's bug list. In the Help menu is a Report a Bug command and a View Bugs List command. These commands open the bug list in your browser.

3. When an important bug appears that we must notify users about -- and when new versions of the application appear -- we send email to a few mailing lists and post on ranchero.com. In addition, the feed for ranchero.com is a default for NetNewsWire users, so they have an easy way to get news from us.

4. Also in the Help menu is a Mailing List command, which takes you to where you can sign up for a NetNewsWire mailing list.

5. We don't send unsolicited email to NetNewsWire users to inform them of bugs or new features, since doing so would probably get our email address placed on a blacklist, and then spam filters would start blocking email from us.

Posted by: Brent Simmons at December 17, 2004 08:53 PM

Mary, you have a perfect right to vent if a product goes wrong. The beauty of blogging is that you have an audience to your venting. You;re not just screaming at a mirror. People can empathise with you, or disagree with you (and thats the way life is). Anyway, keep up the good work. I use Shrook and it suits my purpose, although occasionally it seems to duplicate feeds and I re-read stuff I'm sure I've read before.

Posted by: Morrie at December 18, 2004 12:10 AM

From a developer's perspective there are many reasons why we (especially those in small companies) cannot always fulfill what you expect as a user. I am a user too, but obviously not a typical one, but I do try very hard to work well with my customers, after all they pay for me to buy new techie toys :)

These are, of course, generalisations - there are always exceptions to the rule:

1. Users do not know how to communicate in an effective way with developers. Those of you that have tried before now to build a requirements doc for a customer will know how painful it can sometimes be when customers say things like "I don't care how you do XYZ I just want ABC to happen". That's fine if you really mean it, but I guarantee that when you actually implement there will be more changes. Trying to get more information out of users early on is generally akin to getting blood out of a stone, customers are just not interested in talking detail unless they are already suffering through poor design decisions.

2. Small companies, single developers or MicroISVs or whatever do not always have the resources to manage conflicting feature requests from 3,000 users at once. From my own experience I always need to take a break when the number of feature requests vastly outnumbers the number of bug reports on a released product. What would you as a user expect me to do if three people all ask for conflicting features where there is not a generic solution to please all of them?

3. The vast majority of users never report bugs. A large number of users in this group will also never click 'send error report' if the Application gives them the opportunity.

4. Users have no idea how long it will take to write a particular piece of software. I don't think they should be required to know, but they should be aware that its not always a very quick job, especially when there are bugs to be fixed. In nearly all cases fixing bugs that affect a large number of users is much more important than adding features requested by a small number of people. There is nothing more aggravating than a user asking for a feature and being annoyed when it is not available the next day (worse when the user is your boss). The number of times I've heard a user say "I'd like this application to be able to answer all my telephone calls, it should only take a day or two".

I should say after all of that, which might sound a bit anti-user that I love interacting with users. Hacking the non-technical aspects of a project can be just as much fun as churning out code, sometimes more so. More often than not its certainly more of a challenge ;)

I'm looking forward to reading the corresponding user's post, and I hope that someone is working towards a developer charter that people can sign up to ;)

Posted by: Ross at December 18, 2004 02:15 AM

Hi Mary, thanks for posting the follow-up, I'll certainly point to it.

And I've been on both sides of this, and I know how it feels when people say bad things about a product I put a ton of work into, and am proud of, and seems to do the job for most people who use it.

On the other hand, I flew cross-country twice in the last few weeks, and one time had a great experience, and the other time not so great, and in both cases it was the airline that made the difference. If I were the owner of the airline that wasn't doing so good, I'd want to know what went wrong. But there was no way to tell them.

I've been writing for years that users must demand control of their data, without that it simply won't happen. Products must be rated not only by how fast they are, and feature-rich, and yes bug-free, but also by how little they lock you in. Because lock-in guarantees poor customer service, late-fixes on bugs, a languid market. Without lock-in, users have choice, and power, and the products ultimately will work better.

One of the things that came out of BloggerCon II is the idea of a user's union. Like many ideas hatched at a conf, it didn't go anywhere, but I think it's worth pursuing, if only by shining light on stories like yours.

BTW, a bug report for you Mary, perhaps to pass on to the vendor of your blogging tool. Even if I check Yes to "Remember personal info?" your tool doesn't remember, and I have to re-enter the data every time I post. I can work around it, but it sure is annoying.

Posted by: Dave Winer at December 18, 2004 06:29 AM

Fascinating thread. Here are a few things that I've learned.

1) It never occured to me that archived feeds was user data. I suspect that most NNW users are like me and besides a few flagged posts, they think of this data as ephemeral—like the html of a visited web page. What I do consider to be "my" data is my subscription list. That is easily exportable. But its the people who push their tools that make the progress in the world, so I won't complain.

2) Very few people realize that developers make shitty software. All developers make shitty software. If someone had posted to Dave Winer's classic (http://davenet.scripting.com/1995/09/03/wemakeshittysoftware) on this topic it might have headed off some of the name calling. If users wanted rock solid instead of new features then maybe software wouldn't be so shitty. But as Mary's usage attests, we are still in the era of figuring out what software can do. New features will rule for a while still.

3) Brent Simmons is amazing. What a pro. I am pretty sure that what Mary is doing (or trying to do) with a *beta* version of his software is something he never considered when designing or coding NNW. The fact that it scaled at all to this level is a testament to good design,but I am more impressed with his grace and clear commitment to making his software better. Way to go Brent.

Posted by: Tim at December 18, 2004 07:20 AM

Mary,

As the lead developer of an aggregator (free, BottomFeeder), I have a couple of thoughts. Software that has a data store does need to do backups (in fact, I've just addressed a bug in that part of my application). However, it's also the case that users need to do backups. Why?

Application level backups will likely be to some auxiliary directory. That's not safe enough - if you lose the drive, the primary and backup data both go out the window. That kind of safe backup is out of the hands of developers - we don't know (and can't know) whether you use tape backup, CD's, DVD's, RAID... whatever.

If the data is critical to you, you really need a user level backup scheme - to prevent massive data loss in the event of a hardware failure.

Posted by: James Robertson at December 18, 2004 07:44 AM

I think this brings up some interesting roles of responsibility for these types of distributed apps. If the problem in this was really caused by Technorati's (free) bad feed then how responsible is the maker of (not-free) software which relies on this feed? I don't know the answer but I'm sure it's going to keep coming up more as these types of apps become more prevelant.

Posted by: Ian Landsman at December 18, 2004 07:56 AM

Okay, here is something that is extremeley frustrating for users:
In this circumstance with NNW and Technorati feeds, I've been told by first by Dave and then by Brent that the problem resides with the other. Dave says nothing has changed at Technorati. And in fact, in Schrook, the Technorati feeds are not overwriting the old data so that only the last couple of days show. Schrook is showing old and new data for the Technorati meta feeds. I didn't have Engadget in there several months ago, so I don't have old data there to see if it would have overwritten that data. Brent says it's a date problem that he can do a bug fix on to accomodate what he sees as a Technorati feed problem that changed the past few days.

As a user, I don't really care to find fault, I just want the system to work and not overwrite older data. So who do I go to, to get resolution for this problem? Whom do I believe?

More generally, users often feel caught in the middle between say, the OS maker and some software that sits on the top, or between the maker of the hardware and the OS or apps, where each blames the other. What should users do, what is the responsibility of the software maker, service provider, hardware maker, or OS maker?

Posted by: mary hodder at December 18, 2004 08:10 AM

Four and a half years ago I wrote this:

penmachine.com/2002/10/back-to-archives.html

"...it's sadly no surprise that unless they're selling you a fairly high-end server, computer and operating system manufacturers, including Apple (who should really be ahead of the game here) don't make it easy to reliably back up your system out of the box. Yeah, they'll chuck in a DVD drive and a free movie, but how about [...] a Backup Configuration Assistant that runs the first time you fire up the machine?

"Palm, for one, has the right idea. Every time I HotSync my PalmPilot (vintage 1997), it backs up almost everything to my desktop Mac. If I lose, destroy, or accidentally wipe out the handheld, I can still get at the data on it."

I'm still waiting for desktop OS makers to catch up here. For now, we continue to have to learn the hard way about backups ourselves, and I get paid to do seminars like this:

penmachine.com/eac/2003-11_meeting

Posted by: Derek K. Miller at December 18, 2004 08:50 AM

A very interesting thread. I am both a software developer and a user, so I can empathize with both sides. A few points I'd like to throw into the mix.

* I have recently starting using MacOS X, and I have been impressed by how many applications bundle a trivial-to-use software update checker/downloader. It is a nice feature. However..

* The Internet has made software so easy to patch that I believe developers are much more comfortable releasing low-quality software, and software has never been high-quality to begin with.

I think it is a good time for developers to remember the poor folks who use their software, and build in a bit more robustness, failsafe, and feedback features.

Posted by: James Schultz at December 18, 2004 09:01 AM

I want to reinforce Dave Winer's observation about the customer always being right. I think it is a mistake to take that literally and discuss about degrees of fractions of being right. It is powerful to look at the customer as always being right: Have it that the customer is telling you something you need to know whenever a customer communicates with you. Look at it as a gift.

Also, those suppliers who own a customer's problem as their own and work toward mutual resolution are going to provide the kinds of moments-of-truth that will bring referrals and satisfied customers. And sometime you kiss each other goodby. That can be a successful resolution too. But you want the customer to be satisfied with the outcome, at least knowing that you did everything you could to have it work out and to learn everything there was to learn.

Posted by: orcmid at December 18, 2004 11:23 AM

There's something that struck me, watching the back-and-forth here.  First, there is a tendency to work out protocol interoperability by trial and error over the web, until the misunderstandings are hammered out and people learn what the mistaken interpretations are or there's some really tight validation and certification process.  I don't know if that happened here (and I have never read the RSS specifications, so this comment is not about whether they are tight or not because I don't know). We're still struggling with that around e-mail, so I would think that there should be no surprise to developers that the use of newer multi-party distant arrangements can be perilous. I don't know how an user would be expected to know that, and I certainly don't know what an user could be expected to do about it.

The larger problem, and it may be related to the first, is that we seldom have any evidence that there is specified behavior of the products we use. What is it intended to do? How is it intended to fail? What does it not attend to that you should know about? All of these things are basically missing, so it is difficult to determine whether we are seeing a problem of ours or the supplier's or something that is meant to be that way, like it or not. When "beta" is slapped onto something, the situation seems to be worse.

Observing the termoil here, I have several notes on what I will do to mitigate this sort of uproar the next time I ship some software. My key observation is that we want to avoid users and developers both learning what the product is by trial and error, concurrently. Dating is difficult enough between two people able to relate directly. It's a rotten model for delivering software and services.

Posted by: orcmid at December 18, 2004 11:46 AM

Uh, I should have used "turmoil" above. And I see something I was grappling with. We do allow users to develop expectations and are not responsible in doing so. When a customer is upset with us, it is valuable to consider what expectation has been frustrated, how we may have contributed to that, and how we can provide a remedy. And if there is no remedy, what can we provide to discourage recurrence?

Posted by: orcmid at December 18, 2004 11:55 AM

Oh. About the topic of this thread. Maybe there is no "social norm" about this. Unless you consider repudiation of all responsibilty and relationship a "social" norm. I am thinking of those shrink-wrapped license that says you own nothing, it is not warranted to be suitable for any purpose whatsoever, and if you misappropriate it we will ruin your life. It does kinda seem to be the norm, doesn't it?

Posted by: orcmid at December 18, 2004 12:07 PM

Uhuh - the previous post's comments was another 'let's blame the victim' firing range...from public transport to software development it's very common. To 'lusers' to 'the customer is always wrong' it's endemic.

As a web designer and developer I've seen alpha geek behaviour that would make your toes curl, a total arrogance about people using a website or product and also a very scary 'my information is power so I won't share it or document it or help you' sort.

The best critique I've seen of that is Le Tigre (the band's website) who explains that audio geeks/engineers can come in various types, some who won't share information or help and you need to ride over them. Weirdly I don't see much of this in the software/blogging world, so I think your post is very thought-provoking and good.

And all credit too Brent for trying to sort it out, and posting gracefully. I hope it's sorted...

I write this being outside the RSS or Feed Aggregator software world really, I don't know what the software does. I have though been at the end of really shitty customer service, not just the usual 'it's wrong and it's your fault' all-ends--in a stinking pile method, but also companies that make it difficult to even BUY their service (are you listening TELEWEST?). It's sad when you can't buy something because they can't update their address database!?!? Or their clever software spots a 'double-booking' and helpfully cancels it - when it wasn't :-(

So these issues do come up for the user - whether it's the phone bill, broadband, credit card, whatever. And if users and developers did sort this out I'd guess there'd be less people going postal...cos life is stressful as it is.

And don't blame the user. They pay your wages? And I broadcast any really bad service far and wide for many years...even phoned Oftel about the Telewest thing. I can understand when one bad thing goes wrong, like in this case, but I've had experiences of a 'house of cards' of errors and blunders which avalanche.

And that case hell IS too good for them.

Posted by: Tim at December 18, 2004 08:13 PM

Good to have this discussion.

The following seems to me as close to a universal truth as it's possible to get : "the user is responsible for backing up."

This is NOT negotiable; not because developers are bad or want to duck out of our responsibilities, but because any application is only part of an ecosystem on your machine. And no developer (not even Microsoft) has control over the whole thing. So management of the whole (and responsibility for it) can only belong to the user.

OK, but who's fault is it that users don't feel this responsibility, and don't do it? I'm not sure blame applies. But everyone could have a role to play to help overcome it.

Developers certainly. Every application should have a "back-up" option on the file menu - which puts either the month's work, or the project, or the totality of the data managed by the application, into a single, easily identifiable file, which the user can then copy elsewhere and from which the application could automatically restore it's current state.

Other parts of the software supply chain could help too. More prominent warnings in the documentation and at the bottom of every advert : "Our software may lose your data"

And finally bloggers like you. Having brought up this discussion, and having thought it through, (and having heroically overcome personal pain and pride ;-), you could publicly and unambiguously state that "Yep, the major problem here is I didn't keep good enough backups. Everything else would have been less bad, if I had"

(BTW : this isn't meant to be a cheap shot. Putting this coda on the discussion would really help.)

Posted by: phil jones at December 19, 2004 11:23 AM