• Posted on October 26, 2020
• Unique Submission Team

A History of Social Impact Analysis

The Social Impact Statement is the main written work that students must complete when carrying out a Social Impact Analysis exercise for a “real-world” client.

The idea for the Social Impact Statement originated with Ben Shneiderman’s 1990 address [33] to the Computers and Society special interest group of the Association for Computing Machinery. Shneiderman proposed a model based on the environmental impact statement that would enable software designers to investigate the social impact of the systems they designed in time to incorporate changes in those systems as they are built.

Over the next few years, this idea became the basis for a teaching tool in systems design courses and social and ethical issues in computing courses.[18], [34]

Students Need to See Issues in Real World Problems

The idea for including a Social Impact Analysis exercise in a class on social and ethical issues in computing came from a desire to have students see these issues as real problems that they will have to face in their careers.

With some guidance from the instructor, students will have a chance to locate these problems within the complexity of the technical issues of the system. SIA provides students with the skills of locating these issues and thinking carefully about their ethical import [22], [18].

Three Basic Tools for Investigating

Students are trained in three basic tools to use in investigating the social context within which a system is used:

• Interviewing
• Field Observation
• Day in the Life Scenarios

Students are also guided by the instructor in identifying the stakeholders and ethical issues that are relevant for their particular implementation. In doing this, they use the framework from ImpactCS as a prompt in analyzing the ethical issues relevant at various social levels in the implementation.

Students Prepare a Professional Level Report for Clients

Students search the literature based on the situation (e.g. medical systems, computer supported cooperative work, computer-aided manufacturing, etc.) and on the ethical issues they have identified as relevant. Students prepare a report for their client that includes:

• an executive summary
• a description of the system (physical, logical, procedural, and social)
• an analysis of the ethical issues (stakeholders, principles, risks, etc.)
• recommendations for actions, with analysis of the possible outcomes
• a reader’s guide to literature that will help the clients understand the issues in more depth
• an appendix that describes the methods they used to collect their data and prepare their analysis.

This list covers almost all of the curriculum knowledge units in the ImpactCS recommended curriculum. It does not mention the experience students receive in thinking about the technical design of the computing systems they encounter, or the overlapping benefits this experience will have in helping them to understand issues in Computer-Human Interaction or Systems Design.

By the time students have finished this process they have:

• made a report both to the class and to the client
• gained skills in acting in a professional manner toward real clients
• learned to identify and analyze ethical issues
• designed and negotiated alternative courses of action
• gained an understanding of the social context in which a computer system is used
• communicated their understanding of these issues both orally and in writing.

What is a socio-technical system?

You have divined by now that a socio-technical system is a mixture of people and technology. It is, in fact, a much more complex mixture. Below, we outline many of the items that may be found in an STS. In the notes, we will make the case that many of the individual items of a socio-technical system are difficult to distinguish from each other because of their close inter-relationships.

Socio-technical systems include:

• Hardware Mainframes, workstations, peripheral, connecting networks. This is the classic meaning of technology. It is hard to imagine a socio-technical system without some hardware component (though we welcome suggestions). In our above examples, the hardware is the microcomputers and their connecting wires, hubs, routers, etc.

• Software Operating systems, utilities, application programs, specialized code. It is getting increasingly hard to tell the difference between software and hardware, but we expect that software is likely to be an integral part of any socio-technical system. Software (and by implication, hardware too) often incorporates social rules and organizational procedures as part of its design (e.g. optimize these parameters, ask for these data, store the data in these formats, etc.). Thus, software can serve as a stand-in for some of the factors listed below, and the incorporation of social rules into the technology can make these rules harder to see and harder to change. In the examples above, much of the software is likely to change from the emergency room to the elementary school. The software that does not change (e.g. the operating system) may have been designed more with one socio-technical system in mind (e.g. Unix was designed with an academic socio-technical system in mind). The re-use of this software in a different socio-technical system may cause problems of mismatch.

• Physical surroundings. Buildings also influence and embody social rules, and their design can effect the ways that a technology is used. The manager’s office that is protected by a secretary’s office is one example; the large office suite with no walls is another. The physical environment of the military supplier and the elementary school are likely to be quite different, and some security issues may be handled by this physical environment rather than by the technology. Moving a technology that assumes one physical environment into a different environment one may cause mismatch problems.

• People Individuals, groups, roles (support, training, management, line personnel, engineer, etc.), agencies. Note that we list here not just people (e.g. Mr. Jones) but roles (Mr. Jones, head of quality assurance), groups (Management staff in Quality Assurance) and agencies (The Department of Defense). In addition to his role as head of quality assurance, Mr. Jones may also have other roles (e.g. a teacher, a professional electrical engineer, etc.). The person in charge of the microcomputers in our example above may have very different roles in the different socio-technical systems, and these different roles will bring with them different responsibilities and ethical issues. Software and hardware designed assuming the kind of support one would find in a university environment may not match well with an elementary school or emergency room environment.

• Procedures both official and actual, management models, reporting relationships, documentation requirements, data flow, rules & norms. Procedures describe the way things are done in an organization (or at least the official line regarding how they ought to be done). Both the official rules and their actual implementation are important in understanding a socio-technical system. In addition, there are norms about how things are done that allow organizations to work. These norms may not be specified (indeed, it might be counter-productive to specify them). But those who understand them know how to, for instance, make complaints, get a questionable part passed, and find answers to technical questions. Procedures are prime candidates to be encoded in software design.

• Laws and regulations. These also are procedures like those above, but they carry special societal sanctions if the violators are caught. They might be laws regarding the protection of privacy, or regulations about the testing of chips in military use. These societal laws and regulations might be in conflict with internal procedures and rules. For instance, some companies have implicit expectations that employees will share (and probably copy) commercial software. Obviously these illegal expectations cannot be made explicit, but they can be made known.

• Data and data structures. What data are collected, how they are archived, to whom they are made available, and the formats in which they are stored are all decisions that go into the design of a socio-technical system. Data archiving in an emergency room it will be quite different from that in an insurance company, and will be subject to different ethical issues too.

Socio-Technical Systems change over time

So far, we have been talking about differences between different socio-technical systems. In this section we address the changes that can occur over time within any particular socio-technical system.

An STS is configurable in all its elements, and this allows for change over time. By configurable, we mean that particular items in an STS can change over time, and that even among those items the configuration of one element may change. For instance, the particular mix of hardware and software within an elementary school’s computing lab may change as the school gets access to the internet, or as more teachers begin to use the lab for their classes. But this change might also be reflected in changes in procedure (e.g. rules about access to certain sites) and people (someone may need to take the role of censor in approving or disproving sites) and data (downloaded software, music, cookies, etc. on the machines hard drives).

Change in an STS has a trajectory.

As the above example indicates, the change from a stand-alone computer lab to a lab connected to the internet may produce a coordinated set of changes within the socio-technical system. This coordinated series of changes in an STS is called a trajectory. These changes can occur at the level of the STS itself, as in the internet example, or they can occur at the level of the individual parts of the system. For example, a different (but overlapping) socio-technical system supports the rapid evolution of microcomputers and their regular obsolescence. Elementary schools that do not keep up with this trajectory of the hardware in their system will find themselves quickly beset with problems.

These trajectories are most influenced by those with social power.

Since these trajectories are coordinated, who coordinates them? Research by psychologists, sociologists, and anthropologists in social informatics has led to the conclusion that trajectories are most influenced by and usually support those with social power. A few minutes reflection will make this statement seem self-evident. Social power if measured by money, influence, and other forces available to actors to help them influence change in a way that is in line with their goals. So, saying that trajectories are most influenced by those with social power is saying, in essence, that those with social power have power to influence trajectories. Not too surprising.

But the point is more than this. Trajectories usually support the status quo—those who already have power in a system. These are the individuals who get most influence in the construction of the technical specification of a technology, who pay for its implementation, and who guide its use so that it serves their purposes.

There is still an ongoing debate among those who study such things about whether social power always wins in the contest to influence technological trajectories. There is, for instance, clear evidence that struggle groups, groups with much less political power than governments, have been able to effectively use computing technology (specifically the internet) to augment their power. On the other hand, many repressive governments use technology in finely crafted ways to control the information their populations may access.

Research on the use of technology in organizations has not supported earlier claims that the technology itself will produce a “leveling effect” in organizations (ref to Attwell & Rule). The idea, appealing at first, was that since technology enables easy communication among all levels of an organization, it will have the inevitable effect of “flattening out” the hierarchy in organizations that adopt it. By and large, this has not turned out to be true. Organizations can adopt computing technology with the intent of flattening their hierarchy, and it will help do this. But organizations can adopt computing technology with the intent of strengthening the hierarchy (by, for example, installing keystroke level work monitoring on all computers). Again, it is the socio-technical system that produces the effects and structures the ethical problems, rather than the technology alone.

Trajectories are not value neutral.

A moment’s reflection should lead you to the conclusion that trajectories are rarely value-neutral. Trajectories have consequences and these consequences may be good or ill (or good AND ill) for any of the stakeholders in a socio-technical system. This is why ethical reflection should be a part of thinking about socio-technical systems.

Socio-technical systems and our ethical cases

Why should we use the language and approach of socio-technical system in analyzing our cases? There are really two questions here:

• What does socio-technical analysis add to the standard software engineering approach? Standard software engineering approaches certainly focus on hardware, software, and procedures and rules that should be incorporated into the software. To the extent that they concentrate on people and roles, they are mostly interested in the explicit interaction a person has with the technology and on the place in the hierarchy the person occupies as a result of their role. The concentration here is most clearly on the visible and the documented. A socio-technical analysis adds to this picture those aspects of work that are implicit, not documented, and based in informal political systems and actual (rather than ideal or documented) work practices. For the purpose of designing systems, a socio-technical analysis adds to standard concerns of efficiency concerns about skill development and social context. For the purpose of ethical analysis, a socio-technical analysis adds a concern for discovering the hidden practices and possible undocumented effects of a technological implementation.

• How does socio-technical system analysis differ from standard analysis in ethics? Standard stakeholder analysis in ethics spends most of its time looking (surprise) for stakeholders. This is really only one element of what we have identified as a complex socio-technical system. Procedures, physical surroundings, laws, data and data structures, etc. all interact to structure the particular ethical issues that will be relevant to a particular socio-technical system. Thus, a socio-technical analysis provides a more comprehensive and system oriented approach to identifying ethical issues in a particular implementation of technology.

About Social Impact Statements

The idea for a social impact statement originated with Ben Shneiderman’s 1990 address to the Computers and Society special interest group of the Association for Computing Machinery. He proposed a model based on the environmental impact statement that would enable software designers to find out the social impact of the systems they design in time to incorporate changes in those systems as they are built.

Since then the SIS has caught on as a fine teaching tool at several institutions across the country. Faculty at the Unversity of Maryland, George Washington University, and the University of Texas at Austin have all used variations on this approach to help students understand the impact of the software systems they design.

Why include SIS in the Curriculum?

The idea for including an SIS in a class on social and ethical issues in computing came from a desire to have students see ethical and social issues as real problems that they might have to face in their careers. By identifying these problems in real systems–with some guidance from the instructor–they will have a chance to locate them within the complexity of the technical issues of the system.

This exercise provides students with the skills of locating these issues and thinking carefully about them, and the luxury of having the time to do so. The social impact statement is designed as a class project, to be done by teams of students, looking at real computing systems on or near campus.

These skills could be taken out of the classroom and profitably applied to systems design, and in fact this class project is designed precisely because many authors believe these skills will be useful in system design. But this description limits itself to the use of the SIS as a teaching tool.

The Scope Of The Project

As the major project paper for the Technology and Society class, students at St. Olaf College were asked to do a social impact statement on a real system on campus. With a little telephone work, the students found 12 systems on campus with faculty or staff who were willing to help them understand the systems they had in their charge.

The students called the faculty or staff in charge of these systems the students’ “clients” and students were encouraged to deal with them in the professional manner that such a relationship suggested.

Some examples of the systems include: Computer Assisted Instruction or CAI programs in the medical school, the card access system to the dorms, cafeteria, and library, the database maintained by the health office, the modem pool, the campus public information system, and an emergency room patient intake system.

The list of relevant issues for any client’s system was generated from the topic space suggested by the ImpactCS report on teaching social and ethical issues in computing [19]. This list provides a good starting place for matching social and ethical issues with the concrete aspects of the system under study. The list is obviously not comprehensive, but does serve a useful function in beginning an inquiry.

In addition to using this list to identify issues, students were also acquainted with Collins and Miller’s Paramedic method [9] that begins with a comprehensive listing of the stakeholders involved in any ethical decision. Armed with a list of stakeholders for a system and a list of possible social and ethical issues, students can usually generate a quite plausible and comprehensive set of concerns with which to begin their inquiry.

Some of the systems were quite complex, and suggested more social and ethical issues than could be dealt with in a single class project. The campus public information system, for instance, brought up issues of privacy, reliability, property rights, the use of power, honesty, and a host of other issues associated with networks.

Students needed to quickly focus on one or two of these issues in order to make any headway in such a large topic space. One of the criteria used to help narrow the scope of an issue was a quick look at which issues had been given the most attention by the clients. Students were then encouraged to concentrate on those issues their clients had not considered.

Other methods also exist for paring the list of issues to consider. For instance one can investigate only those issues that seem the most dangerous, or those in which the client is most interested, or those that it would be the easiest to investigate. Of course the criteria one might use in a class project (e.g. can it be finished by novices in a few months?) will differ at times from those one would use in industry.

Goals Of The SIS

There are several goals of an SIS, and they flow from the practical nature of the undertaking. An SIS is not an attempt at publishable social science research, or a theoretical deconstruction of the meanings inherent in a computer system (though it may borrow from both of these approaches). It is a tool to allow students (and hopefully systems designers) to locate and deal practically with the social and ethical implications of the technology with which they are working.

A primary goal is to provide surprises about how the system works and the consequences of its operation, information that is not included in the standard story of how the system works. Safety engineers call these latent errors [31], because they lie unnoticed in a system until an accident or critical incident arises.

The idea of latent errors need not apply to only safety issues. It can apply as well to issues of equal access, privacy, property rights, quality of life, or any area where hidden or unnoticed aspects of the system can pose ethical challenges. Thus, the methods used to produce an SIS must be apt for uncovering these surprises–simply reading the specification sheets will not do.

In addition, the practitioner doing an SIS must be aware of the political issues involved in pointing out surprises to designers, managers, and operators of a system. Even if it is your job to do so, one must be politic in pointing out oversights to people you will need to trust to implement fixes.

A secondary, but still important, goal is to give the clients some practice in thinking about the ethical and social aspects of their own system. In a way, the process of thinking about the issues is as important as the product [37]. Even an extended inquiry into a system cannot be sure of turning up all the issues (or even the most important ones).

Sensitizing the designers, managers, and operators of a system to the social and ethical issues, and giving concrete examples of potential difficulties in the system, may make them more aware of the issues and more likely to detect other issues when they arise.

One way of doing this sensitizing is to provide clients with a document that will be useful in future modifications of the system, and that will point them to the literature describing the particular problems they face.

Creating the Social Impact Statement

The SIS class project is based on library and empirical research, presented in class, and revised in the final documents.

In their library research, students are asked to locate discussions of the likely ethical and social issues associated with their client’s system. As an entry to the literature, they should use whatever references their clients suggest, selections from the set of readings for the course [7], and other references that the instructor or other faculty members provide. They may expand on this reference base using standard bibliographic tools.

One purpose of the literature search is to help students compile a set of readings to recommend to their clients, but it is also helpful to prepare them for the kinds of issues they will be dealing with in the empirical stage of their projects.

Students are asked to use 3 different empirical techniques to locate and analyze the ethical and social issues:

• Interviews with Principle Informants
• Field observation of the system in use
• Construction of day-in-the-life scenarios

The purpose of using different techniques is both to triangulate on issues and to attempt to uncover hidden inconsistencies or oversights that might be unquestioned if only one technique was used.

Interviews with Principle Informants

Students are given practice in taking an expert’s verbal description of a system and unpacking it to determine the criteria the expert was using to describe the system. They are also given practice in constructing an interview protocol that led logically from basic issues to critical functions of the system. In addition, the ethical issues inherent in doing interviews (e.g. confidentiality, respect, informed consent) can be discussed.

Groups can be asked to do at least 3 and no more than 7 interviews. Some groups may conduct follow-up interviews. They are asked to interview the designers and managers of the system, but also to include, where possible, lower level operators and clients of the system (and other important stakeholders in the system).

These multiple views of the system can help piece together a more complex picture of the social and ethical concerns than a view from only one perspective.

Field Observation.

Give students practice in designing coding systems and taking field notes for careful, real time observation of the system in use [see 11 & 14 for suggestions]. The purpose of these observations is to allow students to get a better feel for the chaos and complexity of actual system use–as opposed to the description elicited in the interviews.

Groups are asked to do at least 3-5 hours of real-time observation. For some groups (e.g. the modem pool, the card access system) on-line activity reports might already been collected, and these may be used as a part of the field observation. Ask students to be respectful of people’s privacy, but to also attempt to get as wide and varied a set of observations as time will allow.

Day-In-The-Life Scenarios

This technique, taken from human-computer interaction methods [13], involves taking the data from observations and interviews and using it to construct a “story-line” of a unit of the system over a unit of time.

This could consist in tracking the actions of a person over a day (or over a reboot cycle), or tracking of a piece of personal data from its collection to its purge from ../teaching_with_cases/ttreferencesthe system. If the units tracked, and the time over which they are tracked, are chosen carefully, they can point out critical information that may be overlooked in interviews or observations.

The day-in-the-life scenario is a less structured method of what is called “task analysis” in HCI circles [1]. Other task analysis methods (e.g. construction of task frequency tables) would be useful in some projects, but the primary purpose here is to acquaint students with a flexible method to illuminate gaps in their knowledge.

Careful construction of these scenarios allows students to see gaps in the story provided by their current data (e.g. what happens during shift changes?).

The combination of these three methods allows for cross-checking among the various stories offered by each method, and helps to promote a more comprehensive view of the use and operation of the system, and of the procedures associated with system use and operation.

If one were to do an SIS as part of a system design, it might be either more or less comprehensive than the/se, depending on the time, resources available, and the importance of the social and ethical issues involved.

Part of class discussion can center on how to make these difficult choices while still producing a product on time and within budget. These discussions will be more lively after students have some experience with their analysis and thus some idea of the importance of the issues they uncovered and the difficulty involved in the analysis.

Creating the SIS Report

The final Social Impact Statement report consists of 6 sections:

• Executive Summary
• Description of the System
• Analysis of the Results
• Recommendations
• Reader’s Guide
• Methodological Appendix

Students are encouraged to focus on the client as the report’s audience, and to produce a report they feel will be helpful to the client. Because this project will often be the first one of its kind that students are asked to create, we offer some tips for heading off difficulties and sharing reports with clients.

Executive Summary

The executive summary is a page or two summary of the report. It should include a description of the report and of the system, a discussion of the significant issues discovered, and a list of the top recommendations highlighted on the page. Each of these should be keyed to page numbers in the longer report. The idea is to provide a summary that an executive can read in 5 to 10 minutes to get the basic information about the report. Summarizing information in this way is in itself a useful skill for computer science students to learn.

Description of the System

This description should include the physical, logical, procedural, and social elements of the system. The physical structure includes the machines and other hardware involved, the networks, and the physical facilities in which the system is housed (e.g. the offices). The logical structure includes the data structures and software structures involved in the system.

The procedural elements of the system include the ways in which data is gathered, collated, stored, backed up, and reported. They also include procedures for maintenance, repair, and replacement of the system, and any other relevant organizational procedures (e.g. those related to privacy protection or safety). The social elements of the system include a description of personnel and their relationships to each other and to other relevant stakeholders.

Analysis of the Results.

This section includes a discussion of those concrete aspects of the system that lead to specific concerns. These many include any single aspect of the system or interactions between aspects of the system (e.g. procedures that assume technical maintenance even though personnel are not trained). Patterns of use, patterns of oversight or error checking, specific hardware or software concerns, or specific organizational procedures are all candidates for inclusion in this section. The analysis of these specific concerns should highlight the specific risks associated with them, the probability of those risks occurring, and the likely harm or ethical concerns associated with those risks. Finally, the concrete advantages of resolving the specific concerns should be described.

Recommendations

This section should contain a set of recommendations that address each specific concern mentioned in the previous section. There should be at least two action options for each specific concern, and those options should be evaluated in terms of the client’s goals and the ethical or social concerns involved. In most cases the client’s goals will be multiple. For example, maintaining accurate records, guarding privacy, and minimizing cost. The effect of each option on this suite of goals should be noted. The options recommended should be carefully constructed to avoid simple black-and-white choices (e.g. safeguard privacy vs disregard privacy) and to emphasize the best available options for dealing with the issue. Technical fixes (e.g. use a different backup method) should be included as options, but should not be the only options listed. For instance, procedural changes or personnel training could also be recommended.

Reader’s Guide

This should be a prose introduction to the most balanced and readable discussions of the issues that confront the client. It should include at least one item (e.g. a reader or an advanced article) that will serve as a window to further literature. This section can be organized as an annotated bibliography or as a prose review with references.

Methodological Appendix

This section should contain a rationale for the particular methods chosen, and a detailed and concrete description of those methods. The individual interviews should be noted (those privacy issues here are important) as should the specific questions asked in the interviews and any changes made in the interviews to meet needs. The description of the field observation should include a description of, and a rationale for, the observation sites and times. It should also include a description of the significant events looked for, a description of the significant events discovered, and a list of any changes made in the observation protocol made to meet needs. The description of the day-in-the-life scenarios should include a rationale for the choice of those particular perspectives and time frames, a description of the information from which they were compiled (e.g. interviews, manuals, etc.), and finally, the detailed scenarios themselves.

Heading Off Difficulties

Some students have difficulty getting started on these large projects, so it is recommended that you include interim deadline dates. Useful deadlines might be initial contacts, initial literature searches, protocols for interviews and observations, and a first draft of the report.

The interview and observation protocol deadlines are useful because the instructor can use them to make sure the data gathering is well organized and ethically thoughtful. It takes time for students to get used to the difficulty of these issues, and they will need the entire term available to them to gather the information needed, and to collect the data needed to clarify questions that arise late in the process.

Students try to have their data collection done by at least 3/4 of the way through the term. During the write-up phase, they will almost certainly want to go back for some follow-up interviews or observations, and it is good to have some time in which to do these.

Since these are often large projects, students could work in groups to complete them. However, it is reasonable to ask that every student be involved in every phase of the project (e.g. literature search, data collection, analysis, and write-up).

One way to deal with the perennial issue of unequal group participation is to ask students to rate each other (and themselves) on scales for effort, reliability, and quality of contribution. These ratings can then be averaged into each student’s grade. Students who loaf can be pointed out and have their grade altered in this way. It is also useful for the instructor to reserve a “vote” in the rating, in case the entire group colludes in giving each other high (or low) ratings.

Sharing Reports with Clients

There is both a pedagogical and ethical issue involved in determining whether to share with clients the reports students have generated. This is most intense with those sub-par reports that might convince clients that they have no real difficulties to face in their systems. On the one hand a good report can be quite helpful to a client, and it is motivating for students to know that their reports will actually be used by their clients. On the other hand, a poor report might do more damage than good.

One way to resolve this issue is for the instructor to have a meeting with the client after they have read the report, and for the instructor to point out both the strengths and weakness of the report. If time permits, the client’s reaction to the report could even be used as part of the students’ grade.