David Wastell
email: david.wastell@nottingham.ac.uk

In praise of abundance: why individuals matter in Design Science

Keywords: Universals, particulars, theory, Design Science, Plato, individual differences, unit-treatment additivity 1

The Platonic quest for universal principles dominates the mainstream of IS research, typically relegating individual differences to the error term as pet theories and derived hypotheses are put to the statistical test. In design science, this neglect of "particulars" is especially egregious as it wastes valuable information about individuals and their interactions with technology. I present a case study of the design of adaptive automation, which shows how critical such information can be when designing complex IT-based systems. The obsession with theory has gone too far, I conclude; it is time to fight back against the tyranny of universals.

Fig. 1 Results of experiment on decision aids

My line of attack is primarily ontological, beginning with a "frontal assault" on the assumption of "unit treatment additivity" upon which ANOVA fundamentally depends. This axiom assumes that the response (y ij ) of any "experimental unit" (i.e. human subject) i receiving treatment j is simply the sum of the unit's individual unique response (irrespective of the treatment) y i plus an invariant treatment effect t j . In other words:

y ij = y i + t j
This seemingly innocuous equation is deeply problematic. For the present experiment it assumes that there is a mathematical abstraction denoted by y i, , which corresponds to an individual's intrinsic energy efficiency performance, an in-built measurable property like their height. It further assumes an abstract treatment effect, t j, which is identical for every individual, e.g. that the predictive aid improves everyone's performance by the same amount and that this increment is additive. Let me reemphasize, neither y i or t j are in the real world, they are metaphysical inventions, ideal types. Exposed and held up to critical examination, they are deeply questionable, if not absurd. In what sense is the quality of "energy efficiency" meaningful as some constant, intrinsic attribute of an individual? Why should all subjects respond to the same treatment by the same amount? Why is the operation additive; why not some other mathematical form, multiplicative for instance? Why not, indeed! Statistically, ANOVA further assumes that the variation between individuals follows a normal distribution. This is more metaphysics, though empirically testable in this instance. The test does not turn out well: it is patently clear from the figure that the data are highly skewed. 31 of the subjects (approx. 70%) gain a score of over 16, the remaining 14 scatter in a long trail of declining performance. The overall mean1 of 15.8 is empirically meaningless; it sits quite detached from the bulk of the data, in no man's land. It is clear from the graph that most people are about as good as each other in performing the task. Drilling down reveals the true picture, that the minority who struggle with the task are concentrated in the control group. 5 subjects in this group perform very badly, though the majority do as well as the subjects in the aided conditions. This is by no means a simple treatment effect. It suggests that the decision aids are useful but only for the minority of individuals who find difficulty with the task.

These metaphysical reflections beg the obvious question. Why is ANOVA so universally used when it makes such untenable assumptions? The answer: because it provides an expedient way of testing the statistical significance of hypotheses, and thus provides the orthodoxy that most follow. Beyond this critique of the standard ANOVA analysis, I wish to make a further point. Not only is the conventional analysis metaphysically implausible, it is also deeply wasteful. So much critical information has been thrown away, about the particulars of individual variation. Yes, the treatment effect is significant, but it only explains a small proportion of the variation. The partial eta (η2) of 19.5 is hardly a cause for self-congratulation; it means that 80.5% of variation goes unexplained. From a design science point of view, such information really matters. We need to know what works and what does not, why people differ and how technology can be adapted to individuals. This knowledge is critical especially if large investments are to be made in new technology to increase productivity. For the conventional behavioural scientist, in his platonic quest for universal laws, we can understand why particulars are a nuisance, to be consigned to the dustbin of the error term. But for the design scientist, particulars and universals should be of at least equal priority. That is the argument of this paper, to celebrate the ideographic over the nomothetic, and the heuristic over the hypothetico-deductive. I shall use my recent research on adaptive automation to prosecute the case.

Case studyadaptive automation

Over recent years, automation has become a salient area of design research (e.g. [START_REF] Parasuraman | Humans: Still vital after all these years of automation[END_REF] as technological advancement has enabled an increasing number of tasks to be completed by machines that were previously the preserve of humans. Automation designs that flexibly adapt to the needs of the human operator have attracted considerable research interest. Adaptive automation (AA) conveys the idea that tasks can be dynamically allocated between the human and the machine according to operational requirements, with changes in task allocation being based on the human operator's current functional state (Inagaki 2003, Kaber and[START_REF] Kaber | The effects of level of automation and adaptive automation on human performance, situation awareness and workload in a dynamic control task[END_REF]. Changes in task allocations are often described in terms of a shift in the level of automation (LOA), drawing on models of automation as proposed by several authors, most notably the seminal model of [START_REF] Sheridan | Human and computer control of undersea teleoperators[END_REF] which distinguishes 10 LOAs, ranging from full manual control (LOA1) to full control by the automatic system (LOA10). The rationale for adaptive automation is the potential for balancing out variations in operator workload and the research literature distinguishes between two main types of adaptive automation: implicit and explicit [START_REF] Tattersall | Demanding work, technology, and human performance[END_REF]). In the implicit control mode, the machine decides which LOA is the most appropriate; in the explicit mode, this decision is under the jurisdiction of the human. Overall, when compared to static automation, there seem to be benefits of adaptive automation with regard to operator performance, including reductions in mental workload, although there is a considerable degree of inconsistency in the literature (e.g., [START_REF] Inagaki | Adaptive Automation: Sharing and trading of control[END_REF][START_REF] Kaber | Adaptive automation of a dynamic control task based on secondary task workload measurement[END_REF][START_REF] Sauer | A comparison of adaptive & adaptable automation under different levels of environmental stress[END_REF].

The work featured here is drawn from long-term programme of research on adaptable automation involving colleagues at the university of Fribourg, Switzerland. There are three distinctive characteristics of this work: first, that we have used the same computer-based simulation in all the studies; second, its psychophysiological nature; third, that task performance has been assessed under adverse working conditions (created by an external stressor, white noise) as well as the optimal circumstances of the typical laboratory experiment. Four studies have been reported to date. The first experiment compared the benefits of static versus adaptable automation, [START_REF] Sauer | Designing automation for complex work environments under different levels of stress[END_REF]; although a preference for higher levels of manual control emerged, no advantages were found in terms of performance or mental workload. The second experiment investigated different modes of explicit AA: where the operator was completely free to choose, when a prompt was given, and when a decision was forced (Sauer, Kao, Wastell and Nickel, 2012). No salient differences were found between these different regimes. The third experiment compared two modes of implicit adaptive automation (based on decrements in task performance or the occurrence of high demand events) versus explicit AA, where the operator was free to make the change [START_REF] Sauer | A comparison of adaptive & adaptable automation under different levels of environmental stress[END_REF]. The results for performance suggested no clear benefits of any automation mode, although participants with explicit control adopted a more active system management strategy and reported higher levels of self-confidence. In the most recent experiment, the effect of system reliability was assessed [START_REF] Chavaillaz | System reliability, performance and trust in adaptable automation[END_REF]. Three levels of automation reliability was compared: interestingly, although unreliability undermined trust, no effects were found on the actual choice of automation level.

The present study returns to the central issue of paper three, the feasibility of performance-based adaptive automation. This type of AA is based on a comparison between current operator performance and a normative criterion. Although an obvious case can be made for using direct measures of primary task performance to provide this criterion, a strongly advocated alternative is to focus on indirect measures of mental workload, using secondary task methodology. Regarding the latter, models of human performance suggest that performance on secondary tasks is more sensitive to variations in operator workload [START_REF] Hockey | Compensatory Control in The Regulation of Human Performance under Stress and High Workload: A Cognitive-Energetical Framework[END_REF]. This was the approach adopted in that study. It was assumed, on an a priori theoretical basis, that this was the optimal approach; but as we have seen, it failed to confer an advantage. Whether such an approach was the best one, or even feasible, was not evaluated empirically. It is therefore difficult to interpret the above null result. Can it be taken to mean that performance-based AA is not beneficial, or does it simply mean that there was a problem with the particular version implemented in that study. Here we explore this issue in depth, using an heuristic, idiographic approach. The data for this investigation will be taken from the high reliability condition of our last experiment [START_REF] Chavaillaz | System reliability, performance and trust in adaptable automation[END_REF].

Method

A PC-based simulation environment, called AutoCAMS 2.0 (Cabin Air Management System) has been used in all our experiments. AutoCAMS provides a model of a complex process control task, namely the life-support system of a space shuttle. The simulation involves five critical parameters (CO 2 , O 2 , pressure, temperature, and humidity) reflecting the air quality in the shuttle cabin. When functioning normally, automatic controllers ensure that these parameters remain within a defined target range. When a problem develops, a diagnostic support system is available to provide assistance; it is called AFIRA (Automated Fault Identification and Recovery Agent) and provides five different levels of support, ranging from LOA1 (full manual control) to LOA5, where AFIRA proposes a diagnosis of the fault and an automatic procedure for repairing it.

The main interface is shown in figure 2. Operators have four tasks to accomplish. They are asked to diagnose and fix any system disturbances as fast as possible, and to maintain the stability of the system throughout the experimental session by manual control if necessary. In addition to these two primary tasks, operators had to perform two secondary tasks: a prospective memory task, for which they had to record periodically the current level of the N 2 tank, and an annunciator acknowledgement task, which requires them to click a symbol which appears at irregular intervals (on average about 30 s) to indicate the connection between ground control and the space shuttle.

Thirty-nine participants took part in the original, full study (10 females, 29 males), with an average age of 22.8. The present analysis focused on the 13 subjects in the high reliability condition, in which the automatic systems worked perfectly throughout. Subjects attended the laboratory for two sessions, training and testing, separated by a one-week interval. The testing session lasted approximately 2.5 h (with a 15-min break) and consisted of a sequence of two blocks. Each block was 39 min long and contained five fault scenarios.

Results

All control actions performed by the operator and each change in the system are automatically recorded by AutoCAMS for further analyses. The purpose of the present study was to go back to these original log files of individual interactions in order to test the assumption that secondary task performance provides a reliable and effective basis for detecting changes in mental workload. The connection task was used for this purpose as it had shown a greater sensitivity to workload manipulations in the original study (i.e. it showed a stronger effect of fault difficulty); moreover, it required a more frequent response, giving a finer grained level of temporal resolution. MATLAB programmes were written to process these log files to enable the construction of synoptic graphs giving a detailed record of connection task performance across the experimental session. Figure 3 provides an example; the figure also shows the occurrence of 2 faults, when we may presume that mental demands are objectively higher. For the first analysis, the sensitivity of connection task (CT) reaction time (CTRT) was appraised by examining its temporal profile in response to fault states. For each fault, 3 observation points before the fault and 7 points after fault were extracted. The serial positions (SP) before the fault were designated -2, -1 and 0, and the seven points after the fault, 1 to 7. Note that SP0 indicated the last CT task before the fault began, and SP1, the first CT after its commencement; this was because the CT task and the occurrence of faults was not exactly synchronised. To reduce the effect of outlying values, a logarithmic transformation was carried out, as is customary for reaction time data. Following this, CTRTs for the same serial position were averaged, giving an overall time profile for each participant. Such profiles will show clearly whether CTRT provides a sensitive and reliable indicator of the additional task demands imposed by the fault. If there is a sharply defined increase in CTRT, this suggest it could form the basis for effective performance-based AA; if there is no change in RT, or if it is inconsistent across individuals, this suggests it would not be useful. Figure 4 shows the time profiles for 4 individuals. That there was considerable variability in time profiles is shown clearly by figure 4. Examining the whole group of 13 subjects, almost all showed a tendency for CTRT to increase, though the time course and amplitude of the trend varied considerably. The average peak time was 95 seconds into the fault. Only 5 subjects showed a marked increase (> 0.1 log units) immediately after the fault (SP1); for the remaining 8 subjects, the increase was less than this, and in 4 cases it was less than 0.05, or stayed constant.

Although far from convincing, this provided some prima facie evidence that CTRT is responsive to task demands, i.e. to the additional workload putatively associated with fault handling. Whether a reliable detector could be built is another matter. In order to appraise the how well CTRT could perform in this role, a crude "signal detection" analysis was also carried out. A simple algorithm was designed, similar to that deployed in experiment three of our prior work. CTRT values at any instant were compared to the average across the whole experimental session; an anomalous CTRT was held to have occurred when a certain threshold was exceeded based on the stand-ard deviation of CTRT during the baseline period. Two levels of sensitivity were compared: 0.5 and 1 standard deviation from baseline. Three parameters were of interest: the number of Hits, i.e. the detector accurately identified raised workload when a fault was present; the number of False Alarms, i.e. raised workload was detected, but no fault was present; and the number of Misses, i.e. a fault was present, but the detector did not pick up any augmented workload. Two performance indices were derived from these parameters: Accuracy -Percentage of detections that were correct, i.e. hits/(hits + FAs) Reliability -Percentage of faults accurately detected , i.e. hits/(hits + misses)

Fig. 4 Time profiles for found representative subjects

Results are shown in table 1. Detection performance is poor. At the 1SD criterion, although accuracy was high (few false alarms), half the faults were missed, producing a reliability of only 50%. Lowering the criterion had the expected effect of reducing the misses and hence increasing reliability, although only to 68%, but at the same time accuracy drastically declined to less than 50%.

As well as secondary task performance, primary task performance was also examined as a potential detector of workload. The following algorithm was used: an abnormal load was inferred when any system parameter (oxygen, pressure, CO2 etc.) was out of its prescribed range for more than 10 seconds. The results for this detector are also shown in the table. It will be seen that this crude approach performed much better in terms of reliability and accuracy. It is notable that the average latency for picking up a change in workload was 50.3 seconds, which also compares favourably with the time profile of the secondary task.

Discussion

The universals of Plato are tyrants which 'annihilate' particulars.. [START_REF] Feyerabend | The conquest of abundance[END_REF] This discussion is structured in two parts; first, we will reflect on the significance of the specific results of the experiment, and their implications for the design of adaptive automation and for experimental research in this area. I will then return to main theme of the paper, the need to pursue design research in a heuristic mode, which mixes the ideographic and the nomothetic, and to cast off the Platonic yoke.

The results of our experiment are unequivocal. An axiomatic "human factors" principle has been discredited, the assumption that secondary task performance provides a more reliable method for appraising changes in mental workload than primary task performance. This axiom underpinned our choice of the secondary task as the basis of the adaptive automation regime evaluated in experiment 3 of our research programme. Secondary task performance does decline during times of known stress (fault handling) but it is apparent that primary performance is impaired too, indeed it deteriorates more2 . To the central question of whether secondary task performance affords a reliable basis for AA, the answer is a resounding "no". It was impossible to identify in the graphs of individual subjects, a consistent pattern of secondary task degradation. The detector analysis strongly supported this general conclusion, and indeed showed the primary task to provide a mechanism which worked reliably and accurately enough to support a more feasible algorithm. There is a powerful cautionary tale in this story regarding the limits and seductive lure of theory and the need to challenge dogma. Donald Hebb, in my undergraduate psychology textbook, put it pithily: "theory, like rum, is a good servant and a bad masterthis book has emphazised the importance of theory and the importance of not believing it" [START_REF] Hebb | Textbook of Psychology[END_REF].

It is clear from the present study that the use of performance-based methods of adaptive automation is problematic. Alternative methods are available, including the use of psychophysiological measures. Heart rate variability (HRV), for instance, tends to decrease when workload increases [START_REF] Wickens | Attention, Time-Sharing, and Workload[END_REF]. The advantage of these methods is that they do not pose additional workload for operator to manage. However, the reliability of such methods is yet to be shown and they are costly to deploy. And this brings us to another critical point. What is important in a design context is whether a technology will work in practice. This imposes a much higher level of proof than theoretical research. It is not enough to demonstrate statistical significance, that some feature or manipulation has an effect; after all, the power of an experiment can arbitrarily be increased simply by running more subjects. But an intervention which requires 100 subjects before it reveals itself is not an effect which is likely to have any interest in a practical setting. Before making expensive investments in technology, the cost-benefit equation must be favourable; there must be significant gains in productivity, sufficient to justify the investment. We wish the majority, if not all, individuals to improve their productivity, a condition nearly met in our opening vignette which showed the value of predictive aids in managing heating systems; a sizeable minority of individuals were performing below par, and the provision of the predictive aid certainly seemed to help, unlike the additional assistance afforded by the expert system.

The goals of what [START_REF] Hevner | Design science in information systems research[END_REF] call the behavioral science approach to IS research differ significantly from the design science paradigm. For [START_REF] Hevner | Design science in information systems research[END_REF], the aim of "behavioralists" is to develop psychosocial theories which "explain or predict organizational and human phenomena" surrounding the application of technology. Design science, in contrast, seeks to develop a corpus of practically-oriented knowledge through which "the design, implementation, management, and use of information systems can be effectively and efficiently accomplished" (ibid, p. 77). In management science, Van Aken (2005) makes a similar distinction between two forms of research: Mode 1, knowledge for the sake of knowledge, aimed at explanation and description; and Mode 2 which is multidisciplinary and aimed at solving problems. Van Aken goes on to argues for the recasting of management research in the mould of Design Science, rather than conventional explanatory science.

The primary rationale for design experiments, such as the present, is that they generate data from which we can learn to design better. Realistic simulations ("microworlds") like CAMS provide "a valuable tool in the arsenal of design science… for generating realistic behavioral data, testing ideas and developing theory" [START_REF] Wastell | Human-machine dynamics in complex information systems: the "microworld" paradigm as a heuristic tool for developing theory and exploring design issues[END_REF]. But if we are to capitalize fully on this potential, a break with the Platonism of conventional behavioral science is needed. In our search for universals, particulars have been relegated to the error term. Abundance has been conquered [START_REF] Feyerabend | The conquest of abundance[END_REF], but at a price; the wanton waste of important information.

In an early paper using CAMS, I first made the case for the relevance of idiographic analysis in the context of design [START_REF] Wastell | Human-machine dynamics in complex information systems: the "microworld" paradigm as a heuristic tool for developing theory and exploring design issues[END_REF]. The goal of the design experiment, I argued, was heuristic, "theory generating, not theory testing … to use the rich but controllable environment of the microworld to explore complex behavioural phenomena under quasi-controlled [and] ecologically realistic conditions". The idiographic analysis in that earlier study provided some fascinating insights into the difficulties of controlling a system that was complex enough to present a serious chal-lenge. Some subjects succeeded very well but others manifestly struggled. The weak subjects showed a number of common characteristics: some reacted by withdrawing and adopting too narrow a focus; others responded by taking too much on, throwing themselves into excessive manual control. I likened this dichotomy to the typology of "pathological coping behaviours" (encystment and "thematic vagabonding") observed by another investigator, Dietrich Dörner, with a similar passion for particulars rather than universals. Such an idiographic analysis is not merely an anthropological curiosity; it could have practical value too, suggesting alternative options for implementing adaptive automation. Perhaps a qualitative pattern-matching strategy attuned to detecting symptoms of encystment or vagabonding, might well provide a more effective approach than the measurement of simple quantitative properties of performance. This is something to be explored in future research.

Finally, on a statistical note, it is surely time to give up the black magic of the orthodox ANOVA. There are other ways of testing the null hypothesis, without its dubious ontological and statistical baggage. Testing the null hypothesis simply means evaluating the probability that the difference between the three groups of the heating experiment could have arisen by the chance allocation of subjects to groups. A simple randomisation test would accomplish this3 , without making any statistical or metaphysical assumptions. It provides a direct test of what was operationally done in the experimenti.e. individuals were actually randomly assigned to three groups and we have compared their average performance. What we want to know is simply whether the performance of the three groups represents a genuine effect. The mean is just one way of characterizing the overall performance of the group, but it is just that, a humble "real world" summary statistic, not a mysterious "population" estimate, hovering spectrally in the background, like Quetelet's homme type.

6

Coda: down with Plato! Two Methods -Elizabeth [START_REF] Sewell | The Orphic Voice: Poetry and Natural History[END_REF] Doing experiments on people is an odd business. Consider for a moment, the experiment from the perspective of the "subject". Answering the call for participation, you turn up at the laboratory. In this "strange situation", you are instructed what to do, but given little more information about why you are really here. You are not told to conform to the norm, but you're expected to carry out the task like everyone else, to behave like l'homme type. But (unlike the man at the back of the mob in "The Life of Brian") you are an individual and you can only tackle the task in your own way, making sense of what is going on and doing your best. But your individual efforts, however heroic or perfunctory, are of no interest to the experimenter; in his scientific arrogance, only the treatment mean matters, the rest is error, silent error. How very odd is that, you might thinkhe might at least have asked me what I thought of the experiment. It was about the design of a system, and I had quite a few ideas which might have helped improve it… I did get a lot of training, but I still wasn't sure what to do. But you were not paid to think -you have been infantilized, treated like a guinea pig! But it is the experimenter that wears the motley. Better not to do experiments on people; perhaps, better to work with them, with users "as partners and coproducers of design knowledge, rather than passive guinea pigs" [START_REF] Wastell | Homeward bound: ecological design of domestic information systems[END_REF]. In that latter paper, I commented ironically that the current practice of design science, by aping the scientific method, was not itself following well-established precepts of effective design work, e.g. prototyping and user participation. I ended on the chastening thought that, had we worked more collaboratively and iteratively with our users, we may well have produced not only better decision aids (and not spent time and effort on an expert system which users clearly did not like) but more robust theory too.

And a (nearly) final thought. Although my reflections have been directed at the design research, they apply to research in the conventional mode, i.e. behavioural research (Hevner) or mode 1 research, to use Van Aken's terminology. Again, the argument is the samewhat a waste of information not to look into particulars to try to understand patterns of individual variation, seduced instead by our Platonic infatuation with universals. Instead of boasting that a significant correlation has been found confirming a cherished hypothesis, we should be more humble. We like to think that a correlation of 0.3, for instance, is impressive. But a correlation of 0.3, means less than 10% of the variation is explained by our hypothesis. Our ignorance (90%) thus exceeds our knowledge by nearly an order of magnitude. Now the final paragraph in which too much is crammed, but here goes. Any scientific endeavour involves the design of an information system, i.e. a sociotechnical system of people and technology for capturing, processing, and making sense of data. The technology may be very sophisticated, such as the Large Hadron Collider, or mundane, such as a paper-based questionnaire survey. The subject of IS research is IS… so the IS researcher uses IS to study IS; what else was our experiment but an IS. In our perennial angst about the proper subject of our field, some have argued that the technological artifact is what gives IS research its distinctive identity. But this is absurdit arbitrarily removes from our purview any IS not based on computers! For me the considerations raised in this paper are not esoteric debates about how best to carry on our research 4 ; they directly relate to our core businesshow best to design an information system. The matter of what technology to use is secondary; epistemolo-gy5 is at the heart of IS, i.e. considerations of the best means of producing valid knowledge about the world. This applies whether the IS has been developed by an IS researcher to study IS; or it has been deployed by an organisation to manage customer relations. Research expertise is, at bottom, IS design expertise. What then are the implications of my argument for the practical business of developing "real world" IS. Actually in the real world, the particular fares better; there is greater concern with understanding individual variation (e.g. customer segmentation in marketing) though again there is the same hierarchy of knowledge, the same Platonic tendency to endow statistics such as means and correlation coefficients with superior prestige, because they are held to reveal general, universal principles. This reverence has got to stop. Surely it is time to leave the ceremonial table, to shake the tambourine and join the naked dancers! 7

Fig. 2 .

 2 Fig. 2. Main interface of AutoCAMS. Components shown include: (A) history graph for key parameters, (B) functional schema of the cabin (with O 2 /N 2 flow meter readings), (C) target levels of flow, (D) system clock, (E) connectivity check icon (secondary task: probe detection), (F) N 2 tank level logging facility (secondary task: prospective memory), (G) manual control panel, (H) repair facility, (I) subsystem failure indicator, (J) repair status (indicates type of repair in progress), (K) control panel of support system, and (L) the support system information display (AFIRA).

Fig. 3

 3 Fig. 3 Example data for 1 subject showing primary and secondary task changes across time and in relation to the occurrence of faults.

Table 1 -

 1 Detection performance of the various algorithms averaged across the 13 subjects

	Detector	HITS	MISSES FA	Accuracy Reliability
	1 SD, secondary task	5	5	1.4	77.8%	50%
	0.5 SD, secondary task	6.8	3.2	7.2	48.8%	68.3
	Primary task	9.5	0.5	1.3	87.9%	95%

The mean itself is a dubious metaphysical abstraction, if regarded as the estimated property of an unseen, inferred (and therefore unreal) "population", rather than a simple descriptive way of denoting the mid-point of a group of numbers.

In fact, the data of the original experiment show this effect too: two levels of fault difficulty were present; the results showed the effect of novel vs. practised faults was stronger for the primary (F=17.5) than for either secondary task, where a significant effect was only obtained for the connection task (F=4.15).

Such a test was carried for the present dataset. Interestingly, 10,000 replications generated the equivalent of 65 F values greater than 5.1, i.e. a two-tailed probability of 0.0065. This is actually more significant than the ANOVA result, whilst making no assumption of normality.

And we certainly worry about how to do research: hence the "method wars" which rumble on to this day, amongst those with time on their hands! But because we tend to see these issues as ones of research methodology, rather than IS design, we limit the applicability of our expertise to our research practice, rather than applying it to the design of information systems in general.

Of course, to mention epistemology in the context of management is to risk ridicule as an "other-worldy" egg-head, but there have been noble attempts to raise the standard , most notably[START_REF] Stamper | Management epistemology: garbage in, garbage out[END_REF].Comment [w1]:


Paul J Ambrose
email: ambrosep@uww.edu

David Munro
email: munrod@uww.edu

Why Not Let IT Fail? The IT Project Success Paradox

Keywords: IS success evaluation, IT failure, innovation 1

Is a focus on information systems or information technology success a myopic view of evaluating IT success and failure? Are success and failure the opposite ends of a continuum for evaluating IT projects? Conventional measures of success such as meeting cost, time, budgets, and user needs do not address positives that can emerge from failures. We contend that a focus on success and failing to factor the possibility of failure actually hamper IT projects. An organizational mandate that does not allow for failure does not promote risk taking and innovation. It can also foster a project climate fraught with undesirable or unethical behavior and stress among developers, while failing to capture positive lessons that could emerge from IT project failure.

Introduction

Social and business norms expect success at everything we do -at work, at play, and at home. As a society we remember winners and not losers. We do remember failures, though not in the same positive sense as we glorify successes. It is no different when it comes to evaluating, or setting expectations for IT (Information Technology) projects. Success is mandated explicitly or implicitly by project stakeholders. Perhaps that expectation is justifiable. After all business stakeholders invest time, effort and monetary resources in IT projects and as good stewards of business capital, have a rightful expectation of project success defined by the project meeting its set objectives and goals. But then the focus just on immediate project success could be too narrow a world view, and one can even argue that this could be detrimental especially for the overall or long term success of an organization. After all doesn't the old adage "Failure is the stepping stone for success" seem to encourage not willful actions that precipitate failure but the latitude to make mistakes so that we may learn from such errors and consequently become better? Or become more successful in the long run?

The passion to evaluate IT success in IS (Information Systems) academic research, however elusive that evaluation may be, could be traced to early influences on the discipline that originated in the scientific computing foundations and thoughts. Shan-non and Weaver's seminal 1949 work "A Mathematical Theory of Communication" set the stage and standards for information processing with the focus being on the fidelity of information as it moves from the source to destination, and consequently improving technical aspects of communication. At the core of DeLone and McLean's 1992 and 2003 synthesizing work of past attempts to evaluate IS success we find Shannon and Weaver's thesis on technical communication. DeLone and McLean's IS success constructs and their derivatives continue to play a significant role in the measurement of IT success in IS literature. However it is ironic that a model with theoretical roots in science and engineering is attempting to address a problem that is social sciences in nature. The danger of this approach is that we could be missing behavioral factors that need to be addressed ground up in a theoretical model rather than these factors being grafted on top of a scientific model.

One may wonder why such a deterministic technical approach to evaluation IS/IT project success is being criticized, as after all don't we need a systematic approach to evaluate success? The counter to that argument however is twofold. First the time, size, and cost and effort are difficult to determine at the project initiation stage as IT projects follow the "hurricane" model (Dennis et al, 2009, pg 82) where these parameters can be better estimated only when the project is well underway, just as the actual landing sites of hurricanes are known only after landfall. Second, IT has been a key enabler of business transformation, and IT led transformation projects are inherently high risk, high reward projects where the possibility of failure is implicit and integral to the project. Ignoring failure can lead to not learning from such experiences and set the stage for the same mistakes to be committed again. To dig a bit deeper into this subject, let us consider three specific areas where a focus purely on success and failing to factor the possibility of failure in a positive manner is not desirable for IT projects.

2

Central Arguments

Success obsession hinders innovation:

A holistic approach to developing projects and services that meet a myriad of stakeholder needs could be addressed from the perspective of design thinking. Tim Brown elaborates the core principles behind design thinking to include feasibility (it is functionally possible to deliver the needs using technology), desirability (does the project make sense to users) and viability (can it be a sustainable business model). Further, design thinking espouse an organizational culture that accepts "forgiveness" later than asking for "permission" ahead. This is essential to foster innovation by encouraging risk taking which can also lead to mistakes being made. However if the business mandate is for IT projects to meet time, scope and cost, per traditional project management principles, IT professional are likely to adopt a risk averse strategy that curbs innovation. Current IS/IT success models seem to fail to take this into account.

In addition, developers have a disincentive for being innovative in development approaches, methods or tools when success is determined for the traditional perspective currently being followed. In some cases this reluctance to be innovate is because the benefits are not yet well known and may in fact may not exist. The "let someone else try it first" approach is safer. Furthermore changes due to an innovation that is introduced in a process will introduce new risk factors to the project, thus making it harder to evaluate the risk and trade-offs of doing the innovation, which leads to project managers viewing these innovations as unnecessary risks. Perhaps the biggest reason to discourage them is the very fact that implementing the innovation will take time away from the 'actual project'. For example the learning curve alone will likely take more time on this project then the innovation will provide on this project. It is not surprising that often innovation has to forced top down by upper level management, when most often it would be more effective and cheaper for the organization to have it arise organically from the bottom up.

Success obsession can foster unhealthy stress:

Let's start by examining how the stress people feel gets played out. It has to be noted that the stress individual feel about avoiding failure is not uniform and will vary greatly by many factors including their work experience, personality, and factor outside of work to just name a few. However we would be naive to assume that at least some if not most of the individuals on a development effort feel a lot of pressure for a project to succeed, or at least not be the cause of it failing. While the concept of a professional athlete "choking under pressure" has been studied for decades, Sanders and Walia (2012) have shown extended the economic theory that this is not limited to athletes. To quote from their conclusion "Pressure may reduce not only reduce productive output but, more fundamentally, may erode incentives to put forth productive input" While the stress may cause some people to underperform others may leave the organization when their stress levels are too high. As a person's stress from work spills over to their personal lives they are often encouraged by concerned family and friends to consider looking for a less stressful job environment. When this does happen and an individual leaves a project the remaining people on the project are left with more stress from either having to pick up the work of the individual who leaves or in communicating and bringing up to speed the person who replace them. A third way people can deal with stress is to just give up trying to meet what they perceive as unrealistic expectations that are thrust upon them. That is they basically deal with the stress of failure by assuming it is inevitable and hence do what they can but not really wholeheartedly throw themselves into the project.

2.3

Success obsession can motivate undesired behavior: [START_REF] Austin | The effects of time pressure on quality in software development: An agency model[END_REF] highlights the effect of time pressure on developers where developers were known to take short cuts while dealing with unanticipated project complications, even if these shortcuts are not in the best interest of the project. Austin showcases an example where developers deliberately planted a "bug" in the software that would buy them time to do more work in the project when the project "breaks" during production deployment. So holding down people to hard success measures could encourage them to circumvent boundaries that could be considered as being unethical. This is not an isolated report. Zelazny (2011) reported based on a survey that "software development team members do not consider the internal quality attributes of modifiability, portability, and reusability when considering an information systems development project successful." Considering that the development team members are the people who most benefit from these attributes in their day-to-day work shows the degree in which the area takes back seat in pursuit of success.

Conclusion

So where does this all lead to? The bottom line question would be, how do we positively address IT failures as part of IT success evaluation? The motivation is to encourage IT professional to work innovatively and take necessary risks and be proactive contributors on projects. A technical and deterministic view of IS success may not be the right solution to meet that end. Should we then develop a model for IS failure than success?


Darshan Desai
email: dnd@berkeleycollege.edu

Social Software: Silver Bullet or an Enabler of Competitive Advantage?

Keywords: Knowledge, social software, competitive advantage

According to knowledge based view, firm-specific knowledge is considered to be a foundational source of competitive advantage. However, the recent trend towards social software radically changes the very essence, role, and value of firm-specific knowledge in driving competitive advantage. On one hand, firms are very enthusiastic about the investments in the social software initiatives; on the other hand, the broader strategic impact of these initiatives is not clear yet. This position paper highlights deep-seated problems in applying prominent strategic management theories in the context of social software. To address these problems, drawing on literature on value co-creation and service logic, it describes four ways social software can enable a firm's competitive advantage, and call for more research in this area.

Introduction

For years, strategy scholars have discussed how a firm gains competitive success. However, with the fast paced technological innovations, the concepts and frameworks of the past may no longer work today. For example, in the past, scholars [START_REF] Kogut | Knowledge, market failure and the multinational enterprise: A reply[END_REF] have considered tacit and firm-specific knowledge to be fundamental source of competitive advantage. Firms have invested a lot in the knowledge management initiatives that included centrally managed, proprietary knowledge repositories. Today, when the locus of knowledge production is no longer confined within a firm's boundaries, the firm-specific knowledge is no longer sufficient for driving a competitive advantage. Social software supports group interactions, and provides open and inexpensive alternatives for the traditional knowledge management implementations. According to Von [START_REF] Krogh | How does social software change knowledge management? Toward a strategic research agenda[END_REF], these social software implementations raise fundamental questions about the very essence and value of the firm-specific knowledge in driving competitive advantage.

With these technological advancements related to social software, today, there has been a profound shift in the roles of firms and customers in the process of knowledge and value co-creation. Empowered with internet and social media, customers have become more connected, informed, and active than ever before. Scholars [START_REF] Vargo | Evolving to a new dominant logic for marketing[END_REF] have well recognized the shift in the role of consumers. Despite the similar shift in the role of a firm, relatively little is known about it. A few scholars [START_REF] Krogh | How does social software change knowledge management? Toward a strategic research agenda[END_REF][START_REF] Haefliger | Introduction to Social Software and Strategy[END_REF] have provided valuable insights by discussing the role of social software in changing knowledge management and strategic thinking. On one hand, software vendors claim that their social software works like silver bullets that lead to competitive advantages, and, the enthusiasm and investments for these initiatives are on the rise. On the other hand, the broader strategic impacts of these initiatives on competitive advantage are not clear yet.

Social software has potential to empower firms to be more engaged and active; however, it is important to draw on strategic management literature to explore whether and how it enables competitive advantage. This position paper highlights deep-seated problems in applying some of the prominent strategic management theories in the context of social software initiatives. To address these problems, drawing on marketing literature, it suggests four ways social software can enable a firm's competitive advantage and call for more research to extend the dynamic capability view.

2

The Social Software and Strategy Management: Do they match?

In the strategy literature, resource-based view [START_REF] Barney | Firm resources and sustained competitive advantage[END_REF] and knowledge based view [START_REF] Kogut | Knowledge, market failure and the multinational enterprise: A reply[END_REF] places the source of competitive advantage at the organizational level. These scholars argue that internal, difficult to transmit and firm-specific knowledge and resources are important sources of a firm's competitive advantage. However, over time, scholars recognized that a firm's internal stock of resources is not sufficient, and the flow of assets and knowledge resources are equally important for driving a competitive advantage. To acknowledge the importance of the flow of assets and external sources of competitive advantage, [START_REF] Teece | Dynamic capabilities and strategic management[END_REF] extended the resource based view, and presented the dynamic capability approach. According to the dynamic capability approach, when competitive landscapes are shifting, the dynamic capabilities by which managers "integrate, build, reconfigure, internal and external competencies to address rapidly changing environments" become sources of competitive advantage (Teece et al., 1997, p.516). Despite the recognition of the external competencies, due to its roots in resource based view, dynamic capability approach still remains internally focused, and considers enterpriselevel dynamic capabilities to be the sources of a competitive advantage.

In these internally focused views, firm's internal value creation is considered to be disconnected from the market, and the customer-firm interface is considered to be a locus of extraction of economic value [START_REF] Prahalad | The Future of Competition: Co-Creating Unique Value with Customers[END_REF]. Much of these strategy discussions focus on value extraction and capture, not on value cocreation. On the contrary, the social software is the crucial enabler of value co-creation. Social software boosts and facilitates the interaction between employees within and individuals outside the firm, such as members of user communities or customers [START_REF] Haefliger | Introduction to Social Software and Strategy[END_REF]. It can enable customers to contribute to the firm's innovation and product development [START_REF] Franke | The 'I Designed It Myself' effect in mass customization[END_REF]. It can also facilitate users of technology to organize themselves in communities, and innovate independently of manufacturers [START_REF] Hippel | Horizontal innovation networksdby and for users[END_REF]. In this context, to understand the impact of social software, strategy research should broaden the narrow perspective of authoritative decisions about technology adoption and myopic focus of value capture at the enterprise level [START_REF] Haefliger | Introduction to Social Software and Strategy[END_REF]. The strategy research should also broaden their perspectives to shift due emphasis to the value co-creation and capability building at the system level.

Value Co-creation and Competitive Advantage

Effective leveraging of firm' internal resources and enterprise-level dynamic capabilities for superior delivery of customer value (value-in-exchange) is no longer sufficient for gaining competitive advantage. The prevailing view that the value for customers is embedded in the firms' offerings has been challenged by a view that value for customers emerges in customers' spears as value-in-use. Customers use the market offerings, and collaborate with other users, add other resources (goods, services, and information) and skills held by them, and create value-in-use (Gronroos, 2008). Superior value co-creation built up capabilities at the system level, and enhances the fitness of a firm's business ecosystem that leads to competitive advantage. As shown in Figure -1, most research in the dynamic capability literature focuses on enterprise level dynamic capability as source of competitive advantage. To understand the competitive advantage holistically, it is crucial to explore interactions between enterprise capabilities and fitness of business ecosystem. Social Software Now to understand the impact of social software on competitive advantage, it is important to explore how it enables value co creation and value capture at the enterprise level and at the system level. Therefore, the impact of social software on competitive advantage can be analyzed in four different ways described in Table-1.

Table-1: Social Software's impact on Competitive advantage

As shown in the analytical framework in Table-1, future research is needed to understand i) whether and how social software enable co-creation of superior value-in-use at the system level, ii) whether and how social software enable sustaining and protecting value-in-use at the system level, iii) whether and how social software enable value orchestration and co-creation at the enterprise level, and iv) whether and how social software enable competitive value capture and capability building at the enterprise level. This type of research that takes strategic approach on social software can generate valuable practical insights that are much needed by the practitioners and managers.

Conclusion

This position paper highlights a key issue in applying prominent strategic management theories in the context of social software. According to this paper, applying the prominent theories to explore the impact of social software may give a myopic view. Prominent strategy management theories generally emphasize value extraction and capture at the enterprise-level more than value co-creation at the system level. On the contrary, the social software is the crucial enabler of value co-creation within a value network. Hence, to holistically understand the impact of social software on competitive advantage, it is important to explore how social software enables value co creation and value capture at the enterprise level and at the system level both. For this exploration, it is important to enrich the internally-focused prominent strategic management theories with the insights about value co-creation at the system level.

Figure

  Figure-1: Impact of Social Software on Competitive Advantage


Igor Hawryszkiewycz
email: igor.hawryszkiewycz@uts.edu.au

A System of Systems approach to Managing Emergence in Complex Environments

Keywords: Collaboration, Agility, Modeling, Concepts for collaboration, Living Systems Theory

Collaboration now plays an important role in many organizations. Many organizations often see collaboration as a given and provide a myriad of communication tools ranging from e-mail through workspaces to video conferencing. Assumptions are then made that these tools will be used in a productive manner. However, there are now many example of where goals are not achieved through ad-hoc use of technologies as collaboration is often not aligned to business practice, especially to changing business practices. This paper calls for an approach to align technology use to the enterprise practices.

The paper models enterprises as a system of systems where systems are closely integrated through collaborative spaces. These spaces change during system change. The paper provides a set of concepts to describe a system of system and shows how this can be used to align collaboration to the emerging business relationships. The proposed concepts, in contrast to existing methods, place greater emphasis on social structures. They support the idea of a collaborative architecture, which defines the alignment of social collaborative arrangements to business activities through the creation or rearrangement of collaborative spaces. The goal is to get away from traditional approaches in choosing the best pattern based on history, but to encourage design thinking through experimentation at the business structure level. The paper then describes the kinds of tools needed to support modelling based on these concepts.

Introduction

The emerging trend in business and government systems is towards greater networking or what is sometimes called Enterprise 2.0 [START_REF] Mcafee | Enterprise 2.0: The Dawn of Emergent Collaboration[END_REF]. Network arrangements often include a number of organizations, who collaborate to jointly co-create products [START_REF] Cova | Marketing solutions in accordance with S-D logic: Cocreating value with customer network actors[END_REF] or services. Apart from the trends, current literature does not look at the emergence and change of relationships that occur in networked organizations and their impact on collaboration. Emergence and change has been the characteristic of business practice for many years. [START_REF] Ciborra | The Platform Organization: Recombining Strategies, Structures, and Surprises[END_REF] describes change at organizational level and develops the concept of a platform. He also identifies a trend to adopt existing patterns in most change decisions in contrast to the trend to design thinking [START_REF] Martin | The Design of Business[END_REF] now emerging in practice. The idea of organization is now going beyond simply business units; it is also going into social communities, which themselves are increasingly seen as organizations. [START_REF] Ibrahim | Community Technology project in Malaysia: Kodai[END_REF] describe the use of ICT in a Malaysian community. [START_REF] Best | Sustainability Failures of Rural Telecenters: Challenges from Sustainable Access in Rural India (SARI) Project[END_REF] describe how changing client behavior leads to the failure of a community where misalignment between practice and technology grows with changing community practices. In communities, as in many business organizations, there is greater emphasis on social structure (Pralahad and Krishnan, 2008) and collaboration. There are increasing calls [START_REF] Pisano | What Kind of Collaboration is Right for You[END_REF], Patel, 2012) for a more focused approach to create a collaborative architecture to align collaboration to the business process and avoid failure because of lack of collaboration.

The complexity that characterizes continual business change is here seen in a similar way to [START_REF] Merali | Using Complexity Science to effect a paradigm shift in Information systems for the 21 st . century[END_REF]. It does not focus on mathematical solutions but ways to manage the continuous change in business relationships. The paper describes the set of modeling concepts that address these issues. The objective of this paper is to develop a model to encourage design thinking in social context by providing a platform, which focuses on collaborative structure within organizations. It brings together ideas from living systems theory (Miller, 1978, Lane and[START_REF] Lane | Application of Living Systems Theory to the Study of Management[END_REF] and design thinking as a set of interacting spaces [START_REF] Brown | Design Thinking[END_REF]. The goal is to align the spaces to the collaboration that best fits the enterprise ways of working. This is in contrast to many current modeling methods, which focus on the technical structure and thus do not address the increasing role of social relationships in business system evolution.

2

Representing a system of systems

The paper proposes an open systems architecture to provide the semantics for emergence at the system level to define a number of classes of communicating system [START_REF] Miller | Living Systems[END_REF]). An enterprise can then modeled in terms of the classes of systems and emergence can be described in terms of changing system structure.

 A group is a small number of people who have a well-defined goal. This may be a sales team, or a software team writing a program, or a temporary focus group working on a proposal. Thus groups usually address one function that requires one or two levels of decision making.  An organization is something bigger and usually includes a number of groups. It may be a large business unit that develops and sells a financial product. It can include a number of groups as for example a product development group, a marketing group and a client relationship group.  Organization is a generic term and organizations can include other organizations. Thus a bank can include a number of business financial units. The organizational level differs from the group level in that it has more than two levels of decision making.



A community is a more loose connection of people. It may for example be a union within an organization. It may be a professional association, or a business group. It may be an association of people formulating a policy.

There are two more detailed levels. An activity is where a group engages in tasks such as to decide how to arrange a software module. A task is then something that is carried out in an activity such as 'develop a program'.

Figure 1 illustrates these concepts with some simple examples of how the different classes of system can be composed into a system of systems. Figure 1 shows a sales group whose goal is to reach a sales target. This group is part of the sales organization that sets the targets. The sales organization is part of a business system that includes the manufacturing and sales organizations. The system of systems is the top-level of the semantics proposed in this paper.

Each of these social systems is described as a lower level set of concepts. These are roles, shown by black dots, artifacts, shown as disk shapes, and business activities shown as ellipses. Links between roles indicate knowledge flows. The sales organization for example has a role sales manager whose responsibility is to 'manage the sales program'. It also has a role 'salesperson' whose responsibility is to make sales. Figure 1 also shows collaborative group between the sales and manufacturing organizations. The concept boundary role boundary role is used to show social interactions across systems An enterprise can also have internal communities, as for example a union, a professional group or a sporting group. All these systems must work togetherhence the increasing emphasis on collaboration. There is also emphasis on social networking to support knowledge sharing within and between systems. The systems are now very likely to be open systems. That is systems that can respond to unanticipated inputs and can reorganize their activities.

Some readers may see a similarity to context diagrams as each system can be seen as a "process" in structured models. The major difference is that whereas modeling methods such as data flow diagrams focus on process and information flows the concepts here focus on social structures within the system given the more networked environments found in most business systems. There is also a difference in typical representations of organizational structure. Figure 2, for example, illustrates a traditional structure where a business is composed of a sales and manufacturing departments. An alternate representation based on a system of systems is shown in Figure 3.

Figure 2 -Traditional Hierarchical view

In Figure 3 systems are seen as interlinked systems. Here the business is modeled as an organization. Sales and manufacturing are seen as organizations within the business. Collaborations in the structure can also be clearly shown. For example there are two collaborations created between the business and their buyer organizations. The connections are through boundary roles that exchange knowledge between systems and maintain alignment between them.

Figure 3 -An Open Systems View

Defining Semantics of Change

The premise in this paper is that change is primarily driven through actions of people occupying roles in businesses. It usually commences with some new collaboration that addresses a perceived opportunity. The organization structure thus changes as productive collaborative arrangements are identified and become part of normal operations. Change is modeled in the following steps:

 Change commences with the emergence of new collaborative group nominating boundary roles in each participating system. These become virtual roles within a collaborative group,  Roles in existing systems are assigned to take on the responsibilities of the virtual roles,  Self-organization takes place by rearranging information flows in line with the new responsibilities.  Collaborative groups, if found effective can then become new systems that can be combined into the system of systems.

Example of Change -Emerging Network

Suppose two business managers, as shown in Figure 4, find that they can work together. One has the expertise in product development and outsources product installation to a variety of installers. The other is an expert in delivery and installation. By combining they feel they will add new value to their businesses. Partner A will be able to focus on creating innovative products while partner B will use their contacts to distribute these products.

Figure 4 -Starting the Collaboration

In the initial stages the collaboration is simplean interaction between the managers of two businesses who see some benefit in working together. Once agreement is reached a more sophisticated collaboration is proposed. This, as shown in Figure 5, follows the same steps as the previous example but on a larger scale.

 The two businesses together create a collaborative organization through which they communicate.  The collaborative organization has its own roles and levels. There is a coordination group, a work scheduling group and a product planning group. There are two roles, the coordinator and designer with specific responsibilities in the collaboration  Virtual roles are created in the collaborating organization. There are a number of roles (shown by the black circle). Each role is assigned a responsibility and dotted links between roles indicated how roles in the two businesses are assigned to the virtual role.  People assigned to the virtual roles collaborate in carrying out their formal duties.

Hence for example the product designer role is responsible for new product design for partner A whereas the team is responsible for developing the product for partner B.

Design Space for Modeling

The modelling method described here is implemented on the open modelling platform at the University of Vienna. The method known as MelCa allows models to be set up from different perspectives and maintains cross references between models as allowed by the open modelling platforms. The modelling concepts used in the model have been described earlier [START_REF] Hawryszkiewycz | A Metamodel for Modeling Collaborative Systems[END_REF]) and support collaboration. The main concepts are a role (shown by a circle, artefact (shown by the disk shape, and activity (shown by the ellipse). New objects can be easily added to the model and rearranged as needed giving the flexibility to use alternate modelling options.

The basic principle used in the model in Figure 6 is to show each system as an aggregation of level in the square box. They must also create a collaborative environment where they can leave together. There is flexibility in rearranging systems, adding new components, and linking systems through collaborative spaces. The tool is highly interactive and supports design experimentation by providing the ability to quickly rearrange systems and collaboration between them. A more detailed application can be found in Yoo (2011).

Implementation issuescreating platforms

Implementations combine objects to create the collaborative systems using adaptable workspaces. An example of a workspace is shown in Figure 8. Beginning with collaboration [START_REF] Hawryszkiewycz | A Metamodel for Modeling Collaborative Systems[END_REF]) which supports the concepts described earlier. Knowledge workers can self-organize their work in a system by creating new roles or artifacts and rearranging responsibilities for working on the artifacts. They should also be able to create interaction spaces to pursue new and evaluate.

Summary

The paper defined the new characteristics of emerging complex systems and the challenges to be met by new methodologies that will help business analysts to deal with complexity and issues of integration of collaborating businesses. The paper defined a set of concepts to manage change using living systems theory [START_REF] Miller | Living Systems[END_REF] as its framework. The paper then identified the kind of tools needed to support the open modelling method and illustrated with an example.

The semantics focused on emergence, self-organization is simply managed by creating collaborative environments as needed. Process integration is also supported through collaborative environments where people assigned to roles that are linked to roles in the collaborative environment. The future work here is to use the semantics to propose services based on social media to quickly adapt social networking to emerging business structures.

Figure 1 -

 1 Figure 1 -Examples

Figure 5 -

 5 Figure 5 -Communicating Systems -An overall architecture

Figure 6 -

 6 Figure 6 -A representation of business networking using MelCa

Figure 7 -

 7 Figure 7 -A working platform


Ahmed Imran
email: a.imran@adfa.edu.au

Shirley Gregor
email: shirley.gregor@anu.edu.au

Tim Turner
email: t.turner@adfa.edu.au

Curriculum Design and Delivery for E-government Knowledge Transfer in a Cross Cultural Environment: The Bangladesh Experience

Keywords: ICT4D, E-government, Cross culture, Knowledge transfer, Curriculum design

This paper describes a successful intervention to facilitate the adoption of e-government in a least developed country. The action design research project adopted an unusual approach to facilitating e-government based on earlier grounded research that identified particular underlying inhibitors to adoption. In this case, the need to increase the knowledge of the Bangladeshi public servants to the potential and approaches of egovernment was a higher priority than attempting any specific e-government project implementation. A formal education program has been established that is available to all Bangladeshi public servants through institutional training. The project's approach and the lessons learned in its delivery provide a useful framework for other egovernment interventions in least developed countries.

Introduction

In the 21st century's information-intensive economy and society, "knowledge" is considered the most precious commodity, and knowledge transfer initiatives can make significant differences in under privileged societies [START_REF] Kefela | Knowledge-based economy and society has become a vital commodity to countries[END_REF]. The e-government capacity building project described here aimed at building knowledge of the benefits of information and communication technology (ICT) for decision making and work practices among public sector leaders. This two year long project involved action design research (ADR) [START_REF] Iivari | Action Research and Design Science Research -Seemingly similar but decisively dissimilar[END_REF]Venable 2009, Sein, Henfridsson et al. 2011) and application of reflective practices [START_REF] Loughran | Effective Reflective Practice: In Search of Meaning in Learning about Teaching[END_REF] in developing an educational curriculum from the ground up. The intervention, with multi-level stakeholders from two countries with different socio-economic and cultural backgrounds, provided rich insights and lessons on cross-cultural design issues, which could be useful for practitioners and researchers working in developing countries. The paper first outlines the project, followed by its rationale and background, discusses the key challenges, lessons learned and implications for the future.

The Project

In 2008, based on existing research work, the first phase of an e-government capacity building project in Bangladesh was launched with the help of the competitive Public Service Linkage Grant (PSLP) from the AusAID [START_REF] Imran | eGovernment Capacity Building through Knowledge Transfer and Best Practice Development in Bangladesh[END_REF]. The broad objectives of the project were to improve transparency and efficiency in the public sector in Bangladesh with enhanced benefits to citizens. This activity addressed an important problem area identified through prior research, a lack of fundamental knowledge and awareness of the strategic use and implications of ICT systems for government business processes; a major barrier to egovernment adoption in Bangladesh [START_REF] Imran | Uncovering the Hidden Issues in e-Government Adoption in a Least Developed Country: The Case of Bangladesh[END_REF]. This lack of knowledge inhibited responses to a range of other barriers such as poor infrastructure, low socio-economic conditions and lack of leadership. Accordingly, a two phase intervention was planned. In 2008, the first phase aimed to: (1) build a long-term strategy for successful e-government adoption, and (2) overcome knowledge barriers by short-burst specially designed training packages for senior decision makers within the public services.

In 2010, the second phase of the intervention was designed building upon the achievements of the initial activity [START_REF] Imran | Educate to Innovate -Fast Tracking ICT Management Capabilities Amongst the Key Government Officials for eGovernment Implementation in Bangladesh[END_REF]) as a next step to institutionalise a more robust knowledge base amongst the public sector officials in Bangladesh. The Bangladesh Public Administration Training Centre (BPATC) was chosen as the counterpart organisation. BPATC delivers training for both new and more senior government officers from all cadres and services of the government, including police, customs and other affiliated organisations. In the past the only ICT-related course at BPATC, "ICT and E-governance", had covered only introductory and basic skills training for ICT and lacked quality and purpose.

To achieve the overarching goal, the activity focused on building a long-term institutional capacity through a well-designed sustainable knowledge building scheme. A techno-centric approach in the past has been the cause of failures of many e-government initiatives in developing countries [START_REF] Heeks | Most eGovernment-for-Development Projects Fail: How Can Risks be Reduced? IDPM i-Government working paper[END_REF]. The approach in our project, in contrast, emphasized 'human-ware' as the precondition or major driver for successful adoption of egovernment. Often, managers' lack of motivation, fear of the unknown, resistance to change and negative attitudes are significant obstacles for such innovation [START_REF] Imran | Uncovering the Hidden Issues in e-Government Adoption in a Least Developed Country: The Case of Bangladesh[END_REF].

Program Purpose

This program was especially designed to give familiarity with the technical and managerial processes required to successfully execute e-government projects and also to highlight the key aspects of successful project management practices in the context of socio-cultural and organisational complexity. The curriculum provides an opportunity to consider real-life experiences through case studies from developed as well as developing countries. The trainee officers are expected to gather clear knowledge of how e-government operates, an understanding of the roles they could be asked to fill, and insight into the combination of managerial, analytic, and technical knowledge they need if they are to handle e-government projects effectively and with confidence.

The program aimed to develop competency in e-government management focusing on three major areas:

 Knowledge of the use ICT and its benefits in e-government in an LDC:  Knowledge of essential ICT systems and technology;  Knowledge of strategic ICT management.

2.2

Course Design The course was developed following an outcomes-based approach to learning, where learning objectives are specified for each learning component prior to its development. This approach allows both instructors and students to be aware of what is expected from each other and also to design the assessments. In specifying learning objectives, Bloom's taxonomy of learning objectives [START_REF] Bloom | Taxonomy of Educational Objectives; The Classification of Educational Goals[END_REF]) that was enhanced by Anderson et al. [START_REF] Anderson | A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom's Taxonomy of Educational Objectives[END_REF] was used as the primary guide. This taxonomy has six levels of learning and each level (Remembering, Understanding, Applying, Analysis, Evaluating and Creating) has "action" words associated with it, which indicate the explicit outcomes expected in terms of learning.

One of the underlying objectives of the program was to help the public sector officials to develop their capacity to recognise how they could use e-government to create a positive impact in society. Realizing this objective required a series of reflections from different stakeholders that focused on understanding the broader issue of egovernment implementation and the culturally-specific issues of the Bangladesh Public Sector and broader society. These reflections were followed by careful crafting of the course. The course design built on the work in the earlier phase, particularly the short-burst training course, because research has shown that such an approach is effective and relevant in addressing the complexity of embedding innovations in the context of a least developing country (Bangladesh). The curriculum has been designed to harmonise with the counterpart's capabilities and meet priority learning needs of public servants. Training material for use in classroom-based teaching included presentation material, individual, small-group, and whole-group exercises, case studies and review questions (with answers), a short quiz and a long quiz (both with answers), and an instructor's manual (i.e. lesson plan).

3

Project Output/ Deliveries

3.1

The "e-Government Management" Training Package A full-fledged university standard course (equivalent to a 13 week semester long course in Australian terms) was designed through a series of grounded studies and consultations with the counterpart organisation, tertiary education designers, relevant experts and stakeholders. The primary audience for this course is government officers in developing countries who have some managerial experience (mid-level). Accordingly, the perspectives of practising managers were incorporated in the course. It is expected that participants may come from different fields (such as arts, engineering, science, management) and will have some tertiary education, plus basic IT skills and knowledge (for example, word-processing and email).

A Text Book on E-government Management for Developing Countries

A preliminary version of a textbook on e-government in least developed countries (in general, not Bangladesh alone) has been prepared, which follows the format of existing quality texts used in Management of IT courses at the post-graduate level. The final text book is a significant output of the activity as it is the first of its kind for developing countries; consisting of four major parts and sixteen chapters. This text book will be reused by BPATC for ongoing e-government management courses as a main text book and guide to the training.

4

Project evaluation and benefits

The project activities included two phases of evaluation, one before the course and one after the course, using a combination of qualitative and quantitative methods. Below is a discussion on the benefits of the course.

Managers in the Public Sector who have Completed the Training Program have Increased Knowledge of How to Oversee the Use of ICT (e-government) in the Public Sector

This was the significant outcome and major objective of the activity. Participating officers had increased knowledge and confidence, which was clearly reflected in the evaluation carried out after the course. It was further evident through their initiatives in their workplace after the expiry of the course (Source: Students reflection during certificate award ceremony after six months of the 7th ITEG Course, Date 8th Jan 2012). Some participants played a 'champion' role in initiating actual e-government proposals and projects and some had made policy contributions in their respective organisations which were publicly shared in the certificate award ceremony.

A High-quality and Research-based Training Course on E-government Management

The newly developed course is a high-quality first of its kind for developing countries that meets the requirements for public sector officials. This paradigm shift has significantly improved the standard, reputation and outcome of the course. The capacity of BPATC was strengthened as it now has a well-designed educational program available for delivery in an area that is essential for the modernization of government.

Before the intervention, staff at BPATC indicated that they see a strong need for developing public sector managers' knowledge of ICT, but they had neither the expertise nor the resources to develop relevant and required curriculum.

A Targeted Textbook for E-government in Least Developed Countries

Some of the participants of this course were appointed to new positions related to new or existing e-government projects. The textbook, which has been shared amongst many stakeholders, is used as a companion, reference book and for consultation. Use of course material was evident beyond the class room environment.

Follow-up interviews with participants after the program show the textbook is in regular use and is providing assistance in managing e-government projects. According to the Rector of BPATC,

The experience of the last few years has shown that there is a strong hope in initiating change in officers' through information technology and e-governance. With the knowledge and expertise gained from the course the vision of good governance and true citizen engagement can become a reality. (Message from the Rector, BPATC at Students' Magazine of 2 nd E-government Management course, 08 January, 2011)

In addition to the planned output, the project has also contributed in other areas in the form of unanticipated benefits. For example, the application of modern pedagogy, assessment and curriculum design in the course provided much insight and knowledge to the course officials and the instructors who could apply those in other courses within the BPATC. Evidence shows that they are doing this. The project has increased Australia's reputation in Bangladesh through knowledge sharing. Australia's whole-of-government approach to the LDC's development problem in the partner country was well-received. The experience and knowledge from the Australian whole-of-government approach has huge potential in the public sector, with capacity building possible more widely through knowledge transfer.

Challenges Faced

Contextualisation of Contents

One of the greatest challenges was to properly manage the context of the educational material. Many of the concepts and topics used in mature IT organisations in developed countries are difficult to translate or contextualise in the nascent environment of the many developing countries that are at the early stages of IT adoption. Use of terms, which have different connotations in different context and cultures, was particularly noticeable (e.g. speaking of a "business" case when discussing government activity)

Significant attention to these issues was needed to meet cultural norms and customs including the use of particular presentation techniques and the careful use of English language and tone to suit the target audiences in the best possible manner. Effort was made to sift the important from the unimportant and to pick the right tone for target audiences to avoid burdening them too much with information that was not particularly relevant or was too technical or specific for their use.

Generating Interest of the Participants

Student participation was the key to success. Maintaining student interest in a newly-developed subject was challenging, particularly as it was addressing an area where they had a self-reported lack of knowledge. Active participation was achieved through good application of pedagogy which included case discussions, projects, video clips, clear power point presentations and relevant content.

Addressing the Sustainability Issue

Sustainability was a key element in this activity. The activity and its connection to the 2008 project is a demonstration of commitment to sustainability by the project team and sponsor, something frequently noted by its absence in other international donor initiatives. Research shows many of the e-government initiatives in developing countries fail due to the 'one-off type' and 'techno centric' approaches [START_REF] Heeks | Most eGovernment-for-Development Projects Fail: How Can Risks be Reduced? IDPM i-Government working paper[END_REF]. This activity has addressed those deficiencies, the knowledge of which slowly contributing to the future projects, where citizens will be able to reap the actual benefit.

Endorsement of the Curriculum

Endorsement of the course and its proper application in replacing the existing offering in the annual training schedule was most challenging. This was also one of the great achievements. Changing curriculum and the name of the course to "E-government Management" through the high-level standing committee of the BPATC was a critical endorsement of the move to a new approach. This outcome was ensured through discussion with the authority concerned in every phase of the design stage.

Ownership and Maintaining the Interest

A further risk minimization strategy was adopted by keeping the counterpart stakeholders involved in the project at all steps so that a feeling of ownership was developed. It was important to maintain the interest of the stakeholders in the program through regular commitments, communications and timely response and delivery of programs. A preliminary road test was carried out with the development team and members from the counterpart organisation to carry out a requirement analysis.

Maintenance of the Integrity of the Program

Over the repeated offerings of the course within the timeframe of the project, there was a tendency to mix in material from other courses within the BPATC program or for modifications to the content to be made by some external instructors to have material that they were more comfortable to teach. However, the project team was diligent to maintain the designed content and delivery so as not to break the integrity of the program, which could have jeopardised the outcomes and overall learning objectives. We tried to make the instructors understand the importance of keeping the program intact, not only to maintain the chronological understanding of the issues but also in attaining external university recognition in future.

Transfer of Officials

Transfer of officials and instructors who were already trained resulted in the need to train others during the time of the project. These transfers meant duplicated effort that had not been foreseen. However, the effort needed to maintain the base of trained instructors was covered by other contingency plans.

Political Instability

Instability in the political environment is an inherent problem in many LDCs beyond the control and scope of individual projects. Political instability and unrest often hindered the smooth running of the activity, which also created uncertainty for the future. Because of the political instability, a clear, coherent strategy cannot be maintained within the ICT sector. It seems that every five years a new party comes to power and changes all previous plans and strategies. This is particularly true in complex, leading-edge agenda, such as IT adoption and e-government exploitation.

Dealing with Bureaucracy

Dealing with bureaucracy and public sector officials in a developing country is always a challenging task, because of the hierarchy and complexities of the procedures involved, especially in implementing an applied research project from overseas. However, this challenge has been met through connections established through one team member's previous working experience in the Bangladesh Government.

Demanding Time Schedule

Managing the project's schedule and the commitment of all stakeholders to match the BPATC program calendar was also challenging. However, through all-out support and dedication from all involved the project worked out well. The significant time spent on negotiation and correspondence between BPATC and the project team on training, coordination and financial matters at times was extremely demanding.

Lessons Learned

Lessons learned from this project will be particularly relevant and useful for e-government interventions in other developing countries. Methodologically, the study provides an example of ADR in an unusual context, an intervention that is aimed at alleviating a situation at a national level. As e-government is frequently aimed at whole-of-government view, or at least cross-government-agency scope, a positive example of supraorganisational intervention is valuable.

The project has also yielded significant outcomes in the area of new knowledge building. Critical reflection has established a number of design principles for a "Sweet Spot Change (SSC) strategy" [START_REF] Gregor | Designing for a "'Sweet Spot' in an Intervention in a Least Developed Country: The Case of e-Government in Bangladesh[END_REF] for interventions of this type, with the most important principle being to first identify a "sweet spot", a point of maximum leverage, and then act on it. In e-government, the sweet spot may not be in the immediately expected areas of technology or business processes. Particularly in developing countries, the 'people' element of best practice e-government approaches may be the most challenging, and rewarding, area.

The project also demonstrates that e-government in a least developed country is greatly facilitated by the inclusion of a leading participant with a working experience, a well-developed reputation and trust in the context of the public sector environment in the LDC. The 'native' insight was instrumental in addressing the culturallysensitive challenges and also in capturing the underlying important issues that are often overlooked. Good access and personal liaison with the stakeholders of ICT in the country, some national policy makers, and Government agencies were also important.

Another lesson learned was that good practice guides from other countries (case studies, best practices) can be effective in achieving learning outcomes, when adapted to suit the local context. The experience and formalisations of the Australian Government's experience in e-government, in this case, were an excellent starting point for a properly-contextualised delivery of the concepts of e-government to Bangladeshi public servants. Similarly, having the support of two of Australia's leading universities, with their capable researchers to work as a team, provided the means to deliver proper course development skills-transfer to the BPATC, the peak training agency for Bangladesh's public service.

Conclusion

E-government properly implemented offers the potential for substantial benefits to least developed countries, particularly at a whole-of-government level. The project outlined here demonstrates that a thoughtfully conducted project that builds on developed country e-government experience but is sensitive to the cultural and environmental factors present in the least developed recipient country can be very effective.

It was evident from many of the evaluation responses collected after each offering of the course that training of this nature was something new to the participants as it addressed practical issues and needs of the students rather than more conventional 'factory-made' training. Developing and delivering the course was also a rewarding learning experience for the researchers/designers, to be part of such international work aiming to contribute to the societal benefit of a disadvantaged society.

The training package and the text book prepared for Bangladesh address issues and concerns common to other LDCs in the field of e-government. A huge potential exists across many developing countries in the world to benefit from replicating this program with some modification to the country context.

There are a number of potential follow-on activities that can be undertaken based on the success of this activity. For example, offering similar e-government programs and training to other developing countries, expanding and developing longer-term training for the government officers of Bangladesh and elsewhere, and through wider dissemination of the e-government management textbook. Interest shown by other organisations affiliated with civil services, universities and IT industry bodies in other least developed countries is encouraging.

  The offering of the program to train local educators on program delivery as a practical train-the-trainer course has strengthened in-house capacity to run this course after the intervention. Staff at BPATC had the opportunity for personal development by participating in the development and delivery of a leading-edge and innovative program. Three offerings of the program occurred, with the third offering undertaken by BPATC instructors alone. Each offering received positive ratings by participants on completion of the course. Follow-up interviews with participants after the program showed material were assimilated and were being applied in practice. According to one participant, It changed my vision, it cleared my idea about e-government, now I'm confident and motivated for egovernment. It will give me an opportunity to introduce ICT in my organisation (Student feedback, 2 nd E-government Management Course, 08 January, 2011).

Acknowledgements

The support of a number of organizations is gratefully acknowledged, especially AusAID, the Australian Agency for International Development, which was the primary funding body. Support was also provided by the School of Accounting and Business Information Systems at the Australian National University (ANU) and by the Bangladesh Public Administration Training Centre (BPATC). We also acknowledge the students, faculty and management of BPATC who were directly and indirectly engaged in developing and improving the curriculum.