What is Human Computer Interaction and Where does Artificial Intelligence stand in HCI?

From using room sized computers to laptops, tablets, mobile phones, AR(Augmented Reality )/VR (Virtual Reality) and evolving technologies such as GRS (gesture recognition systems) and brain computer interfaces it has been a journey for Human Computer Interaction (HCI) systems [1].

Current hot topics in HCI are AR/VR, speech (Natural language and speech recognition > TTS, STT), gestures sensing/recognition and response, and the cutting-edge brain computer interfaces such as sensors that extends to cortex and deep-reaching sensors[2]. Some companies that work on such sensors are Neuralink, Blackrock Neurotech, Synchron, Emotiv and Precision Neuroscience [3].

Artificial Intelligence keeps a special place in HCI development, AI makes it possible for the computers to recognize speech, gestures and respond to them that will replace mouse, keyboards and other intermediates. 3D interaction via AR/VR collected data combined with AI can create real-life like experiences to a new level. Brain-computer interfaces can not only help people with disabilities but can add a “super-power” to a regular human being by one-to-one real-time information processing. And it all comes down to “How to engage the user?”. Computers, or systems with computing capabilities are all products and services that go into interaction with the users. A comprehensive work by Kevin and Gavin Doherty regarding engagement in HCI discusses the design of products and services where a motivation towards using one of the latter is discussed in depth with a broad perspective[4].

UI (User Interface) Design:

Design of User Interfaces for software is a crucial part of software development and packaging into a useful, functional product. While knowledge of software systems and coding is important for the innerworkings of an application, the end user must be kept in mind for final package which is the user interface.

The Eight Golden Rules of Interface Design

One of the pioneers in the HCI Ben Schneiderman’s “Eight Golden Rules of Interface Design” can be found in his book “Encounters with HCI Pioneers” [5], [5], also in the following article by Interaction Design Foundations article [6] where they also provide a checklist to download for use.

1. Strive for Consistency: terminology, visuals, prompts.
2. Seek universal stability: Guides for novice users, shortcuts for experts.
3. Offer Informative Feedback: Each user action creates feedback.
4. Design Dialogues to yield closure: Initiation to completion each task should be grouped.
5. Prevention and Error handling
6. Allow reversal of actions: example undo button
7. Keep users in control.
8. Reduce short-term memory load: Device remembers earlier actions to ease human brain.

UI design principles are based on computer science (technical functionality for I/O operations and data processing, safety, reliability), human psychology and ergonomics for a productive and enjoyable experience for users. The goal is to create an application that:

• Is easy to learn, use and efficient for the designed task,
• Ensures safety and security,
• Delivers accurate and reliable outcomes.
• Follows ethical guidelines for data privacy, bias mitigation [7]

The psychology of Human Computer Interactions is an important aspect while designing a product. Stuart K. Card, Thomas P. Moran and Allen Newell co-authored the book “The psychology of Human Computer Interactions” [8]. The book discusses the author’s work and accomplishments through Applied Information-Processing Psychology Project (AIP) at PARC. AIPs scope is described as to create an applied psychology of human-computer interaction by conducting requisite basic research within a context of application by the authors. The author’s main goal is to use scientific psychology to create an interface that’s easy to use, efficient, error-free and even enjoyable.

Function: The workflow of the software needs to be efficient and accurate.

HCI main components:

Key components in Human Computer Interaction are:

Identification of the user:

• Demographics: age, gender, location, education, …
• Accessibility: disabilities and following WCAG, ADA guidelines
• User skills: newbie, intermediate, advanced
• I/O (Input/Output) design:
• Input devices: mouse, keyboard, joystick, touch, voice, gestures…
• Output environment: screen, audio, haptics.

Interaction Design:

• User Satisfaction- enjoyable; usability testing, Net Promoter Score
• Efficiency: ease of use, and workflow efficiency
• Task Analysis: understanding user goals and tasks.
• Error Handling: error messages and recuperation methods

Interface Design:

• Visual Design: layout, color palette, standardization for home, office, mobile use
• Feedback and Responsiveness: visual or accessibility cues for interactions
• These bullet points guide the UI developer to ask the right questions about Personas, Use Cases, Emerging technologies, Adaptability etc.

So, to summarize, creating personas for use cases, understanding the tasks to incorporate best known methods and technical requirements, understanding the interaction to create non-technical requirements of the UI and I/O yields the best results.

HCI Tools

Design tools: Menu trees, transition diagrams, state charts.

Implementation tools: Mock-up, Programming frameworks and toolkits, UI generators > AI created.

Evaluation tools: Data logging, Evaluation software, feedback, automated survey

How does Artificial Intelligence fit into the equation?

• AI can create electromechanical feedback.
• Guides the Output modalities for interaction with the user.

Some Interesting Projects to Sneak Peek At:

• Massachusetts Institute of Technology (MIT) HCI
• EECS AT MIT, Artificial Intelligence + Decision-making – MIT EECS [9]
• MIT MEDIA LAB Research — MIT Media Lab, A Bionic Leg Controlled by the Brain — MIT Media Lab[10]
• Defense Advanced Research Projects Agency (DARPA) Communicating with Computers (CwC) , Communicating with Computers (CwC) [11]
• Explainable Artificial Intelligence (XAI), Explainable Artificial Intelligence | DARPA [12]
• Augmented Cognition (AugCog) [13]

References

[1]       B. A. Myers, “A brief history of human-computer interaction technology,” interactions, vol. 5, no. 2, pp. 44–54, Mar. 1998, doi: 10.1145/274430.274436.

[2]       J. A. Jacko, Ed., Human Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications, Third Edition, 3rd ed. Boca Raton: CRC Press, 2012. doi: 10.1201/b11963.

[3]       J. E. Woods et al., “Miniature battery-free epidural cortical stimulators,” Sci. Adv., vol. 10, no. 15, p. eadn0858, Apr. 2024, doi: 10.1126/sciadv.adn0858.

[4]       K. Doherty and G. Doherty, “Engagement in HCI: Conception, Theory and Measurement,” ACM Comput. Surv., vol. 51, no. 5, pp. 1–39, Sep. 2019, doi: 10.1145/3234149.

[5]       B. Shneiderman, Encounters with HCI Pioneers: A Personal History and Photo Journal. Springer Nature, 2022.

[6]       E. Wong, “Shneiderman’s Eight Golden Rules Will Help You Design Better Interfaces,” The Interaction Design Foundation. Accessed: Jan. 02, 2025. [Online]. Available: https://www.interaction-design.org/literature/article/shneiderman-s-eight-golden-rules-will-help-you-design-better-interfaces

[7]       K. Shilton, “Values and Ethics in Human-Computer Interaction,” Found. Trends® Human–Computer Interact., vol. 12, no. 2, pp. 107–171, Jul. 2018, doi: 10.1561/1100000073.

[8]      S. K. Card, Ed., The Psychology of Human-Computer Interaction. Boca Raton: CRC Press, 2018. doi: 10.1201/9780203736166.

[9]       “Artificial Intelligence + Decision-making – MIT EECS.” Accessed: Jan. 03, 2025. [Online]. Available: https://www.eecs.mit.edu/research/artificial-intelligence-decision-making/

[10]     “A Bionic Leg Controlled by the Brain,” MIT Media Lab. Accessed: Jan. 03, 2025. [Online]. Available: https://www.media.mit.edu/articles/a-bionic-leg-controlled-by-the-brain/

[11]     “Communicating with Computers (CwC).” Accessed: Jan. 03, 2025. [Online]. Available: https://www.darpa.mil/research/programs/communicating-with-computers

[12]     “Explainable Artificial Intelligence | DARPA.” Accessed: Jan. 03, 2025. [Online]. Available: https://www.darpa.mil/research/programs/explainable-artificial-intelligence

[13]     D. D. Schmorrow and A. A. Kruse, “DARPA’s Augmented Cognition Program-tomorrow’s human computer interaction from vision to reality: building cognitively aware computational systems,” in Proceedings of the IEEE 7th Conference on Human Factors and Power Plants, Sep. 2002, pp. 7–7. doi: 10.1109/HFPP.2002.1042859.