TY - JOUR T1 - Making sense of AI systems development JF - IEEE Transactions on Software Engineering Y1 - 2024 A1 - Mateusz Dolata A1 - Kevin Crowston AB - We identify and describe episodes of sensemaking around challenges in modern Artificial-Intelligence (AI)-based systems development that emerged in projects carried out by IBM and client companies. All projects used IBM Watson as the development platform for building tailored AI-based solutions to support workers or customers of the client companies. Yet, many of the projects turned out to be significantly more challenging than IBM and its clients had expected. The analysis reveals that project members struggled to establish reliable meanings about the technology, the project, context, and data to act upon. The project members report multiple aspects of the projects that they were not expecting to need to make sense of yet were problematic. Many issues bear upon the current-generation AI’s inherent characteristics, such as dependency on large data sets and continuous improvement as more data becomes available. Those characteristics increase the complexity of the projects and call for balanced mindfulness to avoid unexpected problems. VL - 50 IS - 1 ER - TY - JOUR T1 - MIDST: An enhanced development environment that improves the maintainability of a data science analysis JF - International Journal of Information Systems and Project Management Y1 - 2020 A1 - Jeffery Saltz A1 - Kevin Crowston A1 - Heckman, Robert A1 - Yatish Hegde AB -

With the increasing ability to generate actionable insight from data, the field of data science has seen significant growth. As more teams develop data science solutions, the analytical code they develop will need to be enhanced in the future, by an existing or a new team member. Thus, the importance of being able to easily maintain and enhance the code required for an analysis will increase. However, to date, there has been minimal research on the maintainability of an analysis done by a data science team. To help address this gap, data science maintainability was explored by (1) creating a data science maintainability model, (2) creating a new tool, called MIDST (Modular Interactive Data Science Tool), that aims to improve data science maintainability, and then (3) conducting a mixed method experiment to evaluate MIDST. The new tool aims to improve the ability of a team member to update and rerun an existing data science analysis by providing a visual data flow view of the analysis within an integrated code and computational environment. Via an analysis of the quantitative and qualitative survey results, the experiment found that MIDST does help improve the maintainability of an analysis. Thus, this research demonstrates the importance of enhanced tools to help improve the maintainability of data science projects.

VL - 8 UR - http://www.sciencesphere.org/ijispm/archive/ijispm-080301.pdf IS - 3 ER - TY - RPRT T1 - Manifesto on Engineering Academic Software (Dagstuhl Perspectives Workshop 16252) Y1 - 2016 A1 - Alice Allen A1 - Cecilia Aragon A1 - Christoph Becker A1 - Jeffrey Carver A1 - Andrei Chiş A1 - Benoit Combemale A1 - Mike Croucher A1 - Kevin Crowston A1 - Daniel Garijo A1 - Ashish Gehani A1 - Carole Goble A1 - Robert Haines A1 - Robert Hirschfeld A1 - James Howison A1 - Kathryn Huff A1 - Caroline Jay A1 - Daniel S. Katz A1 - Claude Kirchner A1 - Katie Kuksenok A1 - Ralf Lämmel A1 - Oscar Nierstrasz A1 - Matt Turk A1 - van Nieuwpoort, Rob A1 - Matthew Vaughn A1 - Jurgen Vinju AB - Software is often a critical component of scientific research. It can be a component of the academic research methods used to produce research results, or it may itself be an academic research result. Software, however, has rarely been considered to be a citable artifact in its own right. With the advent of open-source software, artifact evaluation committees of conferences, and journals that include source code and running systems as part of the published artifacts, we foresee that software will increasingly be recognized as part of the academic process. The quality and sustainability of this software must be accounted for, both a priori and a posteriori. The Dagstuhl Perspectives Workshop on “Engineering Academic Software” has examined the strengths, weaknesses, risks, and opportunities of academic software engineering. A key outcome of the workshop is this Dagstuhl Manifesto, serving as a roadmap towards future professional software engineering for software-based research instruments and other software produced and used in an academic context. The manifesto is expressed in terms of a series of actionable “pledges” that users and developers of academic research software can take as concrete steps towards improving the environment in which that software is produced. JF - Dagstuhl Manifestos PB - Schloss Dagstuhl – Leibniz Center for Informatics CY - Wadern, Germany VL - 6 IS - 1 ER - TY - Generic T1 - Motivations for sustained participation in crowdsourcing: The role of talk in a citizen science case study T2 - Proceedings of the Forty-eighth Hawai'i International Conference on System Sciences (HICSS-48) Y1 - 2015 A1 - Corey Brian Jackson A1 - Carsten Østerlund A1 - Gabriel Mugar A1 - Kevin Crowston A1 - Katie DeVries Hassman AB -

The paper explores the motivations of volunteers in a large crowd sourcing project and contributes to our understanding of the motivational factors that lead to deeper engagement beyond initial participation. Drawing on the theory of legitimate peripheral participation (LPP) and the literature on motivation in crowd sourcing, we analyze interview and trace data from a large citizen science project. The analyses identify ways in which the technical features of the projects may serve as motivational factors leading participants towards sustained participation. The results suggest volunteers first engage in activities to support knowledge acquisition and later share knowledge with other volunteers and finally increase participation in Talk through a punctuated process of role discovery.

JF - Proceedings of the Forty-eighth Hawai'i International Conference on System Sciences (HICSS-48) CY - Koloa, HI ER - TY - CONF T1 - Motivation and data quality in a citizen science game: A design science evaluation T2 - Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46) Y1 - 2013 A1 - Kevin Crowston A1 - Nathan Prestopnik JF - Forty-sixth Hawai'i International Conference on System Sciences (HICSS-46) CY - Wailea, HI ER - TY - CONF T1 - Mechanisms for Data Quality and Validation in Citizen Science T2 - "Computing for Citizen Science" workshop at the IEEE eScience Conference Y1 - 2011 A1 - Wiggins, Andrea A1 - Newman, Greg A1 - Stevenson, Robert D. A1 - Kevin Crowston KW - Citizen Science KW - data quality KW - data validation AB - Data quality is a primary concern for researchers employing public participation in scientific research, or “citizen science,” to accomplish data collection and analysis tasks. This mode of scientific collaboration relies on contributions from a large, often unknown population of volunteers with widely variable expertise. In this paper, we review the commonly employed mechanisms for ensuring data quality. We also discuss results of a survey of citizen science projects that reports on the use of some of these mechanisms, noting that it is most common for projects to employ multiple mechanisms to ensure data quality and appropriate levels of validation. JF - "Computing for Citizen Science" workshop at the IEEE eScience Conference CY - Stockholm, Sweden UR - http://itee.uq.edu.au/~eresearch/workshops/compcitsci2011/index.html ER - TY - Generic T1 - Machine Learning and Rule-Based Automated Coding of Qualitative Data T2 - American Society for Information Science and Technology (ASIST) Annual Conference Y1 - 2010 A1 - Kevin Crowston A1 - Xiaozhong Liu A1 - Allen, Eileen E. A1 - Heckman, Robert KW - FLOSS KW - NLP AB - Researchers often employ qualitative research approaches but large volumes of textual data pose considerable challenges to manual coding. In this research, we explore how to implement fully or semi-automatic coding on textual data (specifically, electronic messages) by leveraging Natural Language Processing (NLP). In particular, we compare the performance of human-developed NLP rules to those inferred by machine learning algorithms. The experimental results suggest that NLP with machine learning can be an effective way to assist researchers in coding qualitative data. JF - American Society for Information Science and Technology (ASIST) Annual Conference CY - Pittsburgh, PA ER - TY - Generic T1 - The motivational arc of massive virtual collaboration T2 - Proceedings of the IFIP WG 9.5 Working Conference on Virtuality and Society: Massive Virtual Communities Y1 - 2008 A1 - Kevin Crowston A1 - Fagnot, Isabelle AB - Massive virtual collaborations (MVC) involve large numbers of mostly unpaid contributors collectively creating new content. Wikipedia is the most dramatic example of MVC; smaller-scale examples include blogs and discussion groups and free/libre open source software (FLOSS) projects. In this paper, we propose a model of motivations for contribution to MVC that integrates various theoretical perspectives to extend prior work. Specifically, we distinguish three different levels of contribution to projects (initial, sustained and meta) and capture the dynamic and recursive effects of contributions on emergent individual and project states. JF - Proceedings of the IFIP WG 9.5 Working Conference on Virtuality and Society: Massive Virtual Communities CY - Lüneberg, Germany N1 - Please refer to the more recent version of the paper. ER - TY - CONF T1 - Meet Me in Cyberspace: Meetings in the Distributed Work Environment T2 - Academy of Management Conference, OCIS Division Y1 - 2005 A1 - Chudoba, Katherine M. A1 - Mary Beth Watson-Manheim A1 - Lee, Chei Sian A1 - Kevin Crowston KW - Computer-Mediated Communication KW - Virtuality AB - Meetings have long been a critical activity in contemporary work life. At least since Mintzberg's classic study of managerial behavior, researchers have documented and practitioners have bemoaned the amount of time spent in meetings. Despite these problems, meetings are becoming even more common in organizations. Teams are now found throughout the organization, from the manufacturing floor to senior management. Organizational work increasingly occurs in teams, with participation in meetings becoming a core work activity. How are these meetings conducted? What are the implications of the increased use of ICT in these meetings? As more and more members of organizational teams are distributed and must make extensive use of ICT in order to work together, these questions become even more important. Using the device of genre system, an interrelated set of socially constructed communicative actions, we examine meetings from the perspective of employees at a company in the technology industry. We began with the questions: Are meetings in the 21st century different from Mintzberg's conceptualization? If so, how? Drawing on data gathered from interviews that used entries in the employees' electronic calendar system, we found that employees are attending a large number of meetings (20% reported more than 25 meetings in a week) and spending significant time in meetings (27% reported more than 30 hours in a week in meetings). The majority of meetings included non-collocated participants and extensive use of ICT. We explore the implications of these and other findings for collaboration and ICT support. JF - Academy of Management Conference, OCIS Division CY - Honolulu, HI ER - TY - JOUR T1 - Methods for modeling and supporting innovation processes in SMEs JF - European Journal of Innovation Management Y1 - 2005 A1 - Scozzi, Barbara A1 - Garavelli, C. A1 - Kevin Crowston KW - Coordination KW - Organizational Modelling AB - Purpose – Sets out to investigate business modeling techniques (BMTs) which can be used to support and improve innovation processes within small and medium-sized enterprises (SMEs). Design/methodology/approach – Based on a literature review, different analysis perspectives on innovation processes are identified and discussed, and some firm needs and problems are pointed out. The importance of BMTs to firms is further tested by an empirical study whose initial results are reported. Finally, by matching problems and techniques characterized by the same ontology, the BMTs most suitable to address SME needs are identified and their role within the innovation process discussed. Findings – The main result of the paper is the identification of the problems facing SMEs in innovation processes and the possible support offered by BMTs. Though methods and models alone do not assure the success in the innovation development process (IDP), they are enabling factors and can support the creation of strategies, reasoning, insights and communication. Originality/value – The adoption of such BMTs, facilitating the codification of the characteristics of the IDP, might be particularly useful in those environments where, due to the lack of specialized resources, it is difficult to structure all of the information related to the innovation process and to exploit the related benefits and opportunities. VL - 8 IS - 1 ER - TY - Generic T1 - Market-enabling Internet agents T2 - Proceedings of the International Conference on Information Systems (ICIS) Y1 - 1996 A1 - Kevin Crowston ED - DeGross, Janice I. ED - Jarvenpaa, Sirkka ED - Srinivasan, Ananth KW - Internet Agent AB - The growth of the Internet offers a vision of ubiquitous electronic commerce. A particularly exciting feature of Internet commerce is the ability to automate the search for price or other product information across multiple suppliers by using an “agent” to retrieve the relevant information. The use of such agents has the potential to dramatically reduce buyers’ search costs. We argue that such agents effectively transform a diverse set of offerings into an economically efficient market and that their use should therefore be analyzed in these terms. In this paper, we present a simple model of the competitive effects of agents used to support purchasing. The model suggests that agents can be successful for diversified goods but resisted for commodities and near-commodities. We illustrate our model by analyzing the situation of current electronic commerce ventures on the Internet. JF - Proceedings of the International Conference on Information Systems (ICIS) CY - Columbus, OH, December ER - TY - CHAP T1 - Modelling coordination in organizations T2 - Artificial Intelligence in Organization and Management Theory Y1 - 1991 A1 - Kevin Crowston ED - Masuch, Michael ED - Massimo, G. KW - Coordination JF - Artificial Intelligence in Organization and Management Theory PB - Elsevier CY - Amsterdam ER -