Evaluating Smart Home Services and Items: A Living Lab User Experience Study

Abstract

South Korea is expected to become a super-aged society by 2025, when more than 20% of its population will be aged 65 and over. One possible solution for minimizing the socioeconomic burden posed by this aging trend is smart home technology, which can be used to support older adults’ daily routines. In this study, the aim is to suggest the direction of development on smart home technology and products to enhance our understanding of service and item needs for the optimal commercialization of smart homes for users. A living lab was set up to research the experience of using smart home technology in real environments. To obtain intuitive and specific evaluations, visitors of varying ages tested each item and living space and responded to a five-point scale questionnaire on importance (recognition) and performance (satisfaction). The recognition and satisfaction for each smart home item or service were then evaluated using basic statistical analysis, importance–performance analysis, and factor analysis using SPSS. It was determined that the importance and performance evaluations of smart home services and items differed by age group. The scores for the two categories exhibited evident similarities in the older adult group. More similarities were found in the evaluation of performance than importance across age groups. The results show that different age groups agree that the development of services/items that can constantly and automatically check residents’ health status should be prioritized.

1. Introduction

South Korea is facing a fundamental demographic shift, with exceptional growth in the older adult population. In 2022, 9.018 million people, or 17.5% of the total population, were aged 65 or older [1]. Furthermore, South Korea has been ranked the fastest aging among 11 countries—the others being Japan, Canada, the United States, Italy, Australia, Spain, Germany, France, the United Kingdom, and Austria—with 7–14% (aging society) and 14–20% (aged society) of the population reaching the age of at least 65 in 18 years and 7 years, respectively [2]. According to the National Assembly Budget Office, owing to this rapid population aging, the cost of supporting older adults will increase from 21.8% in 2020 to 58.2% in 2040 [3]. The corresponding increase in pension and medical welfare spending will pose a heavy strain on society. Against this background, researchers and industries are studying various ways to offer older adults daily life support, one of which is the development of smart home technologies that automate the monitoring of daily living and wellbeing [4,5].

The concept of a smart home offering new services can be traced back to futuristic display homes in 1930s America, developed at a time when electricity consumption was unproblematic and presenting [6]:

… unprecedented levels of luxury, relaxation, and indulgence, with excessive the benefits of modern living with less effort from householders [7].

The concept remained a specialized one for some time, only able to take shape for a mass market in the final quarter of the 20th century as computing power became increasingly accessible and automated appliances more commonplace [6].

The barrier of entry for the usage of smart products with the latest technology tends to be high for older adults, whom many companies and studies are targeting as the main user class [8,9]. On the contrary, although the present youth and middle-aged groups can expect to have relatively high levels of familiarity with electronic devices even in old age, investigations into their demands and needs are presently lacking. The current study addresses these issues. The aim is to provide a better understanding of prospective users’ needs for smart homes (services and devices). This study is an attempt to broaden the existing understanding through user evaluations of two categories, focusing on empirical evidence regarding users’ perceptions and not just understanding the preference for smart homes and technology. For smart homes to succeed, we need to understand the recognition and demands of prospective users.

An accurate grasp of the support requirements and preferences of prospective users will help set the direction of development on smart products and technology, minimizing risks for both research and businesses. Thus, it is necessary to prepare for the upcoming aging society through extensive research not only on current older adults but also the middle-aged and youth.

The next three questions identify prospective users’ perceptions of smart homes and broaden the understanding of the direction in which smart home services should develop:

Q1. 

How do prospective users perceive various smart home technologies?

Q2. 

How do prospective users evaluate various smart home technologies?

Q3. 

Is the evaluation of smart home technology aligned with users’ perceptions?

Each of these questions is answered through the analysis of survey data (n = 628). A survey of smart home services and device assessment provides insight into how demand varies by prospective users. It showcases an important perspective in shaping age-friendly smart home services.

2. Literature Review and Methodology

2.1. Smart Homes

The smart home originated from building automation technology. The development of the first smart automation system paved the way for smart homes in 1966 [7]. Indeed, the affordability of innovative technology was improved with the development of the microcontroller in 1971 [10]. Additionally, the introduction of gerontechnology in the smart home field proved a milestone intervention in improving the quality of life of older adults in the 1990s [11,12].

Smart home technology became very popular at the dawn of the 21st century. The advancement in information and communications technology, artificial intelligence, and the Internet of Things (IoT) have resulted in a massive transformation of the housing sector. Nowadays, introducing technological support to homes has become a global phenomenon [13,14]. In the last decades, there have been many discussions regarding integrating smart home and service devices in the smart technology and smart home sector. Many researchers have expressed interest in developing smart spaces in living environments [15,16,17,18].

Despite the increasing number of studies, research in this field is confined to three themes. First, studies do not typically consider the multidimensionality of the concept of the smart home, leading to a one-sided representation of its implications, services, and user segments [4,19,20,21]. Chan et al. (2008) attempted to cover the technical state of various smart home projects and established a comprehensive understanding of the current and future challenges that smart homes and smart technologies brought to users. The authors pointed out the tendency to describe the potential benefits of technology while ignoring the users’ viewpoint and following a product-centric approach. According to them, the prevailing technological focus of existing research explains the low acceptance of smart homes in the market [20,22].

Second, studies tend to examine smart homes from a technological perspective, by focusing on the functions of devices and the infrastructure and architecture of automated homes [23,24,25,26]. Several technological aspects of the smart home are covered in recent papers, including the connection with IoT, reasoning systems, fog-based computing approach, and risks. Recently, one of the main advantages of smart home technologies highlighted across smart home research is the optimization of energy consumption [20,27,28,29,30] and the resulting environmental sustainability [19,31,32]. In particular, intelligent sensors, wireless fidelity, load modeling, real-time systems, and computational modeling are identified as emerging knowledge domains in the field of smart homes [13,33,34,35].

Third, the majority of studies discuss the potential advantages of smart home technology [36,37] while providing little empirical evidence regarding users’ perceptions of challenges and benefits [20]. Despite all the potential and prospective growth, the spread of smart home technology, in general, has been slower than expected [14,27,38,39,40,41,42,43]. Thus, users’ perceptions of smart homes need to be deeply investigated to accelerate their adoption and diffusion [14,20,28,40,44]. An intention–behavior gap has been identified in recent research, meaning there is a lacuna between consumers’ estimation and their actual behavior regarding the purchase or use of smart home technologies [14,41,45,46]. Understanding users’ adoption process beyond perceptions is, therefore, essential [43,47,48]. The author felt the need to include this dimension while conducting smart home living lab research. Therefore, this study is expected to deepen that understanding with the analysis of user perceptions and effects through the experiences of smart home services and devices.

2.2. Materials and Methods

After conducting a primary questionnaire survey regarding simple preferences for smart homes (services and items) [9], we considered a more detailed survey for this study, particularly regarding whether there may be a difference between the evaluation of preference and behaviors in future choices. This study was conducted using survey contents that distinguish importance (recognition of service/item) and performance (satisfaction upon which the choice of service/item is based). In other words, in the importance evaluation, although the user’s general recognition affects the result, their decision regarding actual purchases may be different. Thus, even if the importance evaluation score is high, it may not be a proper user evaluation if only a few of them lead to choose (consumption, etc.).

For this series of studies, a selection of service cases on smart home items within the current technical scope was researched and collected; subsequently, a living lab where visitors could experience and evaluate the use of these items in a space resembling a real residence was created. This is because it can be difficult to perform an actual evaluation of product utility without experiencing its use. This approximately 200 square meter living lab was constructed in April 2018 at a university in South Korea. As shown in Figure 1, it consists of a bedroom, open living room, and kitchen, similar to the spatial structures of South Korean apartments. However, the bathroom was designed in such a way as to differentiate it from existing spaces to enable the proposal of new scenarios. Even though people do not actually reside in living labs, the spaces were planned according to the layout of the average home. The functionality of each space was then matched with applicable smart home services/items. The entire complex space consisted of “residing space+IoT+active smart service,” and the linked items were designed to allow visitors to experience the actual operation of the space. Thus, prospective users’ evaluation was designed to be practical and experience-based to enable the effective assessment and prediction of actual smart service and item demand.

The smart home services and items were based on a thorough investigation of various cases, technologies, and items that have already been developed but have not yet been subjected to universalization. From these, services and items (front door, bedroom, toilet, hallway, kitchen, and living room) were selected based on the characteristics of daily residential behavior (

Table 1. Resident behavior.

Daily Behavior (Indoor)
Eating	Getting Dressed
Meals	Clothes
Snacks	Laundry
Cooking
Rest	Mobility
Sleeping	Walking
Napping	Standing (sitting)
Bathing and Hygiene	Communicate
	Talking
Bathing	Using the Internet
Showering	Other
Washing up	Exercise
Elimination	Study/learning
Cleaning	Taking medicine

). These items and services for a comfortable and safe living can be classified according to four categories (

Table 2. Smart home value classification [9].

Life	Category	Service
Comfort	Daily life support convenience management	Schedule for management and provision of daily life information, such as the weather. Support for various living conditions, such as finding lost items and doing household chores. Coordination of video calls with family and friends. Facilitation of leisure through digital devices. Management of lighting.
Safety	Health care	Body signal detection, such as blood pressure, pulse, and blood sugar. Health measurement and management. Identification of disease and management of diabetes through bowel analysis. Provision of medication dosage and administration times.
	Emergency and safety management	Smart door lock and active emergency response service. Control services for infants, older adults, people with disabilities, and pets.
Economy	Eco-friendly energy management	Smart green home. Pro-energy power generation system. Smart home-based energy and management services.
Joy	Leisure Culture	Bidirectional shopping. Social activity/entertainment.

) devised by the Korea Smart Home Industry Association: comfortable life, safe life, economic life, and joyful life. “Comfortable life” is relevant to convenience and management in daily life; “safe life” is relevant to health maintenance and emergency management; “economic life” is relevant to eco-friendliness, energy production, and efficiency; and “joyful life” is relevant to entertainment, such as sociocultural aspects and shopping.

The living lab items were as follows (

Table 3. Smart items in the smart home living lab [9].

Location	Item Name	Item Contents	Importance (a)	(a) SD	Performance (b)	(b) SD	Energy Optimization
Entrance	(A) Check-in	Where walking aids, essential for older adults, are placed in designated locations, check-in services are provided. The lights are adjusted when people enter the house, and a guiding voice is heard.	4.02	1.119	3.29	1.455	▲
	(B) Last check	Before the user leaves the house, (B) informs the user of an item’s location (w/location sensor) that must be packed on screen at the entrance. The day’s weather and primary schedule can also be checked on screen.	4.43	0.912	4.15	1.141	▲
Bedroom	(C) Light control	When the resident falls asleep with the lights on, the lighting system senses this event and turns off the lights. When a resident goes to the bathroom during sleeping hours, or suddenly wakes up and moves, the light turns on at a muted level, to avoid blinding the resident in the dark.	4.36	0.942	4.22	1.082	▲
	(D) Bed control	(D) is a motion bed, modified to fit the resident’s body shape and posture, and facilitates sound sleep through links with sensors on the ceiling that detect sleep.	4.31	1.000	4.14	1.161
	(E) Curtain control	The sensor recognizes when a resident enters the room and opens and sets the curtain in advance.	3.83	1.141	3.66	1.315
Bathroom	(F) Body composition analysis	When a resident stands in front of the bathroom sink, the floor sensor measures the body composition and compares it with (past) data. Through smart mirrors, the resident can obtain important health information, such as body composition analysis and health care advice.	4.54	0.855	4.36	1.007
	(G) Water temperature color	The sensor visually identifies the temperature, with red for hot water and blue for cold water. The resident can check for their desired temperature visually without touching the water.	3.96	1.167	3.78	1.294
	(H) Smart toilet	The pressure sensor on the floor identifies the user and analyzes their excretion. With information accumulated every day, changes and abnormalities in the body can be immediately identified through smart mirrors.	4.41	0.933	4.27	1.082
Kitchen	(I) Countertop height control	During extended cooking and washing of dishes, the heights of the countertops adjust to suit users based on the floor pressure and the distance from the ceiling.	4.35	0.985	4.15	1.180
	(J) Auto ventilation	When the hood alone lacks ventilation, or if the resident does not detect air pollution, (J) automatically opens the window and operates the ventilation fan.	4.20	1.029	4.08	1.208	▲
	(K) Nutrition	A customized diet is recommended according to nutrition and ingestion conditions and based on food ingredients. (K) shows the nutrition information and recipes of recommended menus on a screen and helps encourage ideal eating habits and nutrition management. Provision of information, such as recipes and nutrients.	4.12	1.033	3.94	1.195
Corridor	(L) Gait analysis	Pressure sensors on the floor, and motion sensors on the ceiling and walls, check the resident’s gait. Users can check information using display items. The resident’s gait width, pressure, and knee angle are checked to help prevent arthritis.	3.97	1.126	3.69	1.331
Living room	(M) Fall management	(M) detects indoor falls and accidents and sends emergency rescue signals to the outside world. In case of an emergency, voice guidance activates a speaker throughout the house, and signals an emergency call for outside help. Emergency signals are sent, together with indoor video information by prior consent, which are linked to emergency number 119, and released from the security system to facilitate rescue efforts.	4.50	0.887	4.13	1.226
	(N) Pill reminder	For proper medication management, the service informs the resident of their medication administration time. The location of the medicine bottle is also illuminated, and guidance on the type of medicine is provided by voice.	4.23	0.993	3.92	1.225
Mean			4.23	1.01	3.99	1.21
Daily life support service and product
Health maintenance service and product
Emergency and safety response service and product

): (1) daily living support services and items: check-in/last check at the entrance; smart lighting, bed, and curtains in the bedroom; water temperature display in the bathroom; and countertop height control in the kitchen, (2) health care services and items: body composition analysis/smart toilet in the bathroom; nutrition care (refrigerator) in the kitchen; and gait analysis in the corridor, and (3) emergency and safety response service and items: auto ventilation in the kitchen and fall management/pill reminder in the living room. Owing to the premise that the services and items in the living lab exhibit functions within a simulated residential space, there were limitations concerning the “economy” (eco-friendly, etc.) and “joy” (leisure, etc.) items among the aforementioned four categories [9].

Importance–performance analysis (IPA) (Figure 2), an evaluation method that simultaneously compares and analyzes the importance and performance (satisfaction) assigned by the user with regard to the evaluation target, was employed. Since its introduction by James and Martilla (1977) as a methodology used mainly in the management field, IPA has been applied in research studies on services [49] and planning elements [50,51]. IPA can help grasp the relative value assigned by the user to the evaluated object, showing the satisfaction and importance of certain factors as perceived by future consumers based on the placement of each factor in one of four quadrants on an XY coordinate plane (Figure 2). Therefore, it is useful for distinguishing underdeveloped elements from those that have undergone excessive development (investment). In particular, it can help determine the areas for future maintenance and improvement through the evaluation of objects that have been used before.

One of the challenges of this study is the evaluation of unfamiliar smart items or services that have never been used. We expected that the public (not experts) would have difficulty in separately evaluating importance and performance. Even after visitors have experienced the use of smart home services and items in the living lab, it would not be easy for them to distinguish between the concepts of importance and performance. Therefore, we attempted to overcome these limitations in the survey by delivering a simple but clear question: to control variables, a survey was conducted on the premise that smart services and items were at an “appropriate price level.”

Q1. 

“How important do you think this item is (for future housing)?”

Q2. 

“Are you willing to install this item in your home?”

The first question, regarding the importance of the product for future housing, can be understood as intuitive and a reflection of recognition by the visitor. The response was used as an indicator of ideal importance and general recognition among different age groups. The second question related to performance and visitor satisfaction (upon which the choice of service/item is based). The visitors would answer the second question by evaluating the product regarding their satisfaction and willingness to pay for it. Thus, in the IPA, the answer to the first question was used as a measure of importance and the answer to the second question served as a measure of performance.

Subsequently, factor analysis was conducted to reveal any differences between the importance and performance (I–P) evaluation. Factor analysis uses mathematical procedures for the simplification of interrelated measures to discover patterns in a set of variables [52]. Attempting to discover the simplest method of interpretation of observed data is known as parsimony, and this is essentially the aim of factor analysis [53]. Factor analysis operates on the notion that measurable and observable variables can be reduced to fewer unobservable latent variables that share a common variance, known as reducing dimensionality [54]. These unobservable factors are not directly measured but are essentially hypothetical constructs used to represent variables [55]. The two main factor analysis techniques are exploratory factor analysis (EFA) and confirmatory factor analysis. Confirmatory factor analysis attempts to confirm hypotheses and uses path analysis diagrams to represent variables and factors, whereas EFA tries to uncover complex patterns by exploring the dataset and testing predictions [52]. EFA is used when a researcher wants to discover the number of factors influencing variables and to analyze which variables “go together” [55]. A basic hypothesis of EFA is that there are “m” common “latent” factors to be discovered in the dataset, and the goal is to find the smallest number of common factors that will account for the correlations [56].

Factor analysis is now used in many fields, such as behavioral and social sciences, medicine, economics, and geography, because of technological advancements [57]. Large datasets that consist of several variables can be decreased by observing “groups” of variables (i.e., factors)—that is, factor analysis assembles common variables into descriptive categories [57]. Factor analysis is suitable for studies that involve a few or hundreds of variables, items from questionnaires, or a battery of tests that can be reduced to a smaller set, to get at an underlying concept and to facilitate interpretation [58]. As it is easier to focus on some key factors rather than having to consider too many variables that may be trivial, factor analysis is useful for placing variables into meaningful categories [57,58]. Many other uses of factor analysis include data transformation, hypothesis testing, mapping, and scaling [58]. The recommended sample size is at least 300 participants, and the variables that are subjected to factor analysis should each have at least 5–10 observations [59]. A factor loading for a variable is a measure of how much the variable contributes to the factor; thus, high factor loading scores indicate that the dimensions of the factors are better accounted for by the variables [57]. In this study, 628 samples were used and all items were analyzed based on a factor loading of 0.40 or more. With factor analysis, it could be inferred which factors and underlying concepts influenced the evaluation of the two questions.

3. Assessment

3.1. Statistical Preparation and Descriptive Statistics

Visitors to the living lab from September 2019 were asked to experience the use of its smart items, and then to evaluate their importance and performance based on a five-point scale. A total of 628 data points were collected, and the visitors were distinguished by age as follows (

Table 4. Characteristics of visitor.

Categories Total		Number of Responses 628	Ratio 100%
Age	20–39	189	30%
	40–59	279	44%
	60+	160	26%

): the young (under 40 years old; 189 data points), the middle-aged (over 40 years old and under 59 years old; 279 data points), and older adults (over 60 years old; 160 data points). Individual or group visitors experienced the living lab based on the instructions of a guide in the order of entrance (in)–toilet–bedroom–corridor–kitchen–living room-entrance (out). After the guided tour, each visitor took a comfortable look at the smart home and had time to manipulate services and items. They then answered a questionnaire at the end of the experiment. In the subsequent analysis of the questionnaire, insincere responses, and those wherein all questions were graded with the same score, were excluded.

As a result of the survey, the Cronbach’s α value of importance was 0.899, whereas that of performance was 0.916, indicating that the items were reliable. The average importance score, for a total of 14 smart home service and item questions, was 4.23 points, based on a five-point scale evaluation. Among these, the highest importance score was 4.53 points for (F) body composition analysis (bathroom), followed by 4.50 points for (M) fall management (living room) and 4.42 points for (B) last-check (entrance). The below-average scores were 4.20 points for (J) auto ventilation (kitchen), 4.12 points for (K) nutrition care (kitchen), 4.01 points for (A) check-in (entrance), 3.97 points for (L) gait analysis (kitchen), 3.96 points for (G) water temperature color (bathroom), and 3.83 points for (E) smart curtains (bedroom). The standard deviation for each item was 0.86–1.17 or less, indicating that the deviations between visitors were not significant and that the visitors had similar experiences and evaluations (Table 3).

Meanwhile, the average performance score, for a total of 14 smart home service and item questions, was 3.99 points, based on a five-point scale evaluation; this score was lower than that for importance. Among these, the highest performance score was 4.36 points for (F) body composition analysis (bathroom), followed by 4.27 points for (H) smart toilet (bathroom), and 4.22 points for (C) light control (bedroom). The below-average scores were 3.94 points for (K) nutrition care (kitchen), 3.92 points for (N) pill reminder, 3.78 points for (G) water temperature color (bathroom), 3.69 points for (L) gait analysis (kitchen), 3.66 points for (E) smart curtains, and 3.29 points for (A) check-in. The standard deviation for each item ranged from 1.00 to 1.45 or less and had a bigger range than that for the importance value. The largest difference between the importance and performance scores was 0.72 points for (A) check-in, whereas the smallest difference was 0.12 points for (J) auto ventilation. When age was not considered, the overall high-scoring items in the importance and performance categories were those related to health maintenance. By contrast, items with low scores were related to daily living support (Table 3).

3.2. IPA

IPA was performed based on the average values, that is, 4.2 and 3.7, of the importance and performance scores, respectively, to analyze the marketability of the services/items in the smart living lab (Figure 3). Quadrant I encompassed the categories of items requiring constant maintenance owing to their high importance and satisfaction: (B) last check, (C) light control, (D) bed control, (F) body composition analysis, (H) smart toilet, (I) countertop height control, (M) fall management, and (N) pill reminder. Quadrant II encompassed high-importance but low-performance items. None of the products and services evaluated in this study were included in this quadrant. Quadrant III encompassed items characterized as having both low importance and low performance: (A) check-in, (E) curtain control, (G) water temperature color, and (L) gait analysis. Finally, Quadrant IV encompassed items with relatively high performance compared to their importance: (J) auto ventilation (boundary line) and (K) nutrition care.

Subsequently, the entire IPA graph was distinguished by age group for a more accurate analysis, as follows.

• Young Group(

  Table 5. Statistics (young group).

  	I-Value	p-Value	I–P
  (A) Check-in	3.7	2.62	1.08
  (B) Last check	4.19	4.16	0.17
  (C) Light control	4.2	4.13	0.07
  (D) Bed control	4.23	4.17	0.06
  (E) Curtain control	3.67	3.48	0.19
  (F) Body composition analysis	4.57	4.32	0.25
  (G) Water temperature color	3.59	3.41	0.18
  (H) Smart toilet	4.23	4.04	0.19
  (I) Countertop height control	4.29	4.13	0.16
  (J) Auto ventilation	4.00	3.92	0.08
  (K) Nutrition care refrigerator	4.01	3.86	0.15
  (L) Gait analysis	3.6	3.14	0.46
  (M) Fall management	4.44	3.68	0.76
  (N) Pill reminder	4.33	3.57	0.62

  )

The average score for importance was 4.07, whereas that for performance was 3.75. In the young group, item categories F, M, N, and I scored high in importance, whereas G, L, E, and A scored low in importance. In the same age group, item categories F, D, B, and I scored high in performance, whereas A, L, G, and E scored low in performance. Although the orders were different, item categories A, L, G, and E remained in the lower ranks for both importance and performance. Furthermore, L did not seem appealing to young visitors. Among all items, it consistently had the lowest score in the I–P evaluations.

For all items, importance scores were greater than performance scores, with A, M, N, and L exhibiting the largest differences between the two scores. The I–P value of these items (A, M, N, L) was more than twice the other values. More specifically, category A was for users of walking aids, M referred to detectors for residents at risk of falling, N was for users who need to constantly take their medicines without forgetting, and G referred to services that check for walking posture. Even though there were no distinctly common characteristics among these items and services, it may be speculated that the users of these items had a high possibility of being older adults. In other words, these can be said to be items with distinct characteristics that support frailness due to gerontification.

Based on the IPA results for the young group (Figure 4), Quadrant I encompassed B, C, D, F, H, and I, which include daily living support services and items and a few for health maintenance. Quadrant II, defined as a top-priority improvement area, encompassed M and N, which are emergency and safety response services and items. Quadrant III, defined as a gradual improvement area, encompassed A, E, G, and L, which include daily living support services and items and a few health maintenance services and items, as in quadrant I. Lastly, quadrant IV, defined as a simple maintenance area, encompassed J and K, which are health maintenance and emergency and safety response services and items.

• Middle-aged Group (

  Table 6. Statistics (middle-aged).

  	I-Value	p-Value	I–P
  (A) Check-in	4.15	3.48	0.67
  (B) Last check	4.44	4.06	0.38
  (C) Light control	4.44	4.27	0.17
  (D) Bed control	4.44	4.21	0.23
  (E) Curtain control	3.97	3.79	0.18
  (F) Body composition analysis	4.62	4.48	0.14
  (G) Water temperature color	4.15	3.94	0.21
  (H) Smart toilet	4.54	4.41	0.13
  (I) Countertop height control	4.44	4.21	0.23
  (J) Auto ventilation	4.27	4.18	0.09
  (K) Nutrition care refrigerator	4.21	3.98	0.22
  (L) Gait analysis	4.20	3.97	0.23
  (M) Fall management	4.54	4.31	0.23
  (N) Pill reminder	4.24	4.03	0.21

  )

The average score for importance was 4.34, whereas that for performance was 4.10. In the middle-aged group, item categories F, M, H, and B scored high in importance, whereas L, A, G, and E scored low in importance. In the same age group, item categories F, H, M, and C scored high in performance, whereas L, G, E, and A scored low in performance. Moreover, for all items, importance scores were greater than performance scores. The lower-ranking item categories for the middle-aged and young groups were the same (A, E, G, L); however, the upper-ranking item categories were different. A, E, and G were the least appealing items for the middle-aged group and consistently exhibited the lowest scores in the I–P evaluations. With regard to the I–P values, A and B exhibited the largest differences, more than twice the other values. Quite notably, the middle-aged and young groups were similar in this regard, that is, item category A had the largest I–P value.

Based on the IPA results for the middle-aged group (Figure 5), Quadrant I encompassed C, D, F, H, I, and M, which included daily living support services and items and a few for health maintenance. Quadrant II, defined as a top-priority improvement area, encompassed B as its sole item. Quadrant III, defined as a gradual improvement area, encompassed A, E, G, L, and N, mainly for daily living support. However, with regard to health maintenance and emergency and safety response services and items, the trends were mixed. Lastly, quadrant IV encompassed J as its sole item, which was also included in the same quadrant for the youth group.

• Older Adult Group (

  Table 7. Statistics (older adults).

  	I-Value	p-Value	I–P
  (A) Check-in	4.15	3.72	0.43
  (B) Last check	4.47	4.28	0.19
  (C) Light control	4.36	4.2	0.16
  (D) Bed control	4.15	3.91	0.24
  (E) Curtain control	3.73	3.62	0.12
  (F) Body composition analysis	4.33	4.19	0.14
  (G) Water temperature color	4.07	3.88	0.19
  (H) Smart toilet	4.34	4.26	0.08
  (I) Countertop height control	4.36	4.05	0.31
  (J) Auto ventilation	4.08	4.08	0
  (K) Nutrition care refrigerator	4.09	3.89	0.2
  (L) Gait analysis	3.94	3.8	0.14
  (M) Fall management	4.47	4.31	0.16
  (N) Pill reminder	4.07	4.12	−0.05

  )

The average score for importance was 4.19, whereas that for performance was 4.02. In the older adult group, item categories B, M, C, I, and H scored high in importance, whereas G, N, L, and E scored low in importance. In the same age group, item categories M, B, H, and C scored high in performance, whereas G, L, A, and E scored low in performance. For this age group, it seemed that the two categories began to exhibit more evident similarities. In the older adult group, the importance value was not always greater than the performance value, unlike in the other age groups. N and J were the item categories with the highest performance values, while E, L, and G were the least appealing and consistently exhibited the lowest scores in the I–P evaluations, as in the middle-aged group. Moreover, A and I exhibited the largest differences in I–P values. Quite notably, all three age groups had this characteristic in common, that is, item category A consistently exhibited the largest I–P value for all groups.

Based on the IPA results for the older adult group (Figure 6), Quadrant I encompassed B, C, F, H, I, and M, whereas quadrant II encompassed none of the tested items. Quadrant III, defined as a gradual improvement area, encompassed A, D, E, G, K, and L, mainly for daily living support. Lastly, quadrant IV encompassed J and N. Noticeably, item category J was consistently in this quadrant for all three age groups.

3.3. I–P Factor Analysis

Factor analysis was used to infer differences in evaluation by age group and varimax rotation was adopted to simplify the factor placement. To broaden the understanding of the difference between I–P evaluations, all items were analyzed based on a factor loading of 0.40 or more. The resulting Kaiser–Meyer–Olkin value was 0.908, and the significant probability for Bartlett’s test of sphericity was 0.000, appropriate for the model [60] (

Table 8. Factor analysis (all age groups).

Importance			Performance
	1	2		1		2		3
(N) Pill reminder	0.734	0.169	(F) Body composition analysis	0.772		0.285		0.009
(H) Smart toilet	0.708	0.246	(H) Smart toilet	0.758		0.185		0.136
(M) Fall management	0.688	0.165	(K) Nutrition care refrigerator	0.659		0.325		0.148
(F) Body composition analysis	0.675	0.256	(N) Pill reminder	0.588		0.108		0.499
(L) Gait analysis	0.642	0.251	(J) Auto ventilation	0.555		0.342		0.223
(K) Nutrition care refrigerator	0.614	0.287	(L) Gait analysis	0.549		0.090		0.449
(E) Curtain control	0.075	0.765	(G) Water temperature color	0.525		0.317		0.299
(C) Light control	0.190	0.742	(C) Light control	0.234		0.748		0.151
(D) Bed control	0.245	0.685	(E) Curtain control	0.148		0.741		0.202
(A) Check-in	0.292	0.534	(D) Bed control	0.430		0.674		0.025
(B) Last check	0.283	00.525	(B) Last check	0.103		0.496		0.457
(I) Countertop height control	0.369	0.512	(I) Countertop height control	0.442		0.494		0.093
(G) Water temperature color	0.469	0.476	(A) Check-in	0.034		0.216		0.812
(J) Auto ventilation	0.443	0.462	(M) Fall management	0.492		0.090		0.653
Extraction method: principal component analysis. Varimax with Kaiser normalization.
Total	% variation	% cumulative	Total	% variation			% cumulative
3.585	25.61	25.61	3.548	25.341			25.341
3.181	22.72	48.33	2.566	18.326			43.668
			1.997	14.263			57.931

).

For All Groups: The results of factor analysis for all visitors are shown in Table 8. As all factors had a load value of 0.4 or higher, they could each be classified as a corresponding factor. With this result, it could be assumed that in their I–P evaluations, visitors perceived two categories related to “health” and “daily living.” Furthermore, health-related (including emergency response) services and items were considered a more significant factor than daily living related. However, J, G, and M were regarded as singularities in the attribute classification part of the factor analysis. In the importance factor analysis, it was understandable for J to be classified with a daily living support item group. On the contrary, the classification of G as a health-related item group was rather ambiguous.

Meanwhile, A and M were classified as third factors relevant to performance. It could be speculated that the third factors (A and M) were designed for older adults: specifically, those who use a walking aid and those who have a risk of falling indoors. In all groups, the results of the factor analysis relevant to importance showed that the classification of health-related (i.e., health maintenance, emergency response) and daily living support items affected the assessment.

Factor analyses for the age groups are as follows:

• Young Group

The results of the factor analysis for the young group are shown in

Table 9. Factor analysis (young group).

Importance					Performance
	1	2	3	4		1		2		3
N	0.736	−0.020	0.234	−0.045	F	0.830		0.087		0.119
M	0.679	0.035	0.108	−0.020	H	0.748		0.189		0.102
K	0.640	0.138	0.134	0.132	K	0.599		0.417		0.218
L	0.613	−0.057	0.260	0.237	D	0.522		−0.080		0.464
J	0.588	0.247	−0.126	0.299	M	0.143		0.817		0.056
E	−0.041	0.780	0.125	0.217	N	0.297		0.724		0.086
A	0.413	0.691	−0.029	−0.202	A	−0.122		0.681		0.255
C	−0.033	0.615	0.119	0.465	L	0.403		0.474		0.082
G	0.109	0.461	0.454	0.159	J	0.381		0.412		0.224
H	0.122	0.177	0.838	−0.031	C	0.017		0.121		0.820
F	0.306	−0.021	0.694	0.140	E	0.088		0.167		0.738
I	0.217	−0.014	0.119	0.761	I	0.255		0.080		0.587
D	0.054	0.290	0.015	0.700	G	0.204		0.281		0.426
Total	% variation		% cumulative		Total	% variation			% cumulative
2.478	19.060		19.060		2.426	18.658			18.658
1.877	14.435		33.495		2.402	18.474			37.132
1.603	12.334		45.829		2.16	16.641			53.773
1.585	12.191		58.019

. As B had a load value under 0.4 in the first analysis, it was eliminated, and the analysis was performed again. This group exhibited the largest number of factors out of the three age groups: a four-factor configuration for importance and three-factor configuration for performance. Daily living support items were demonstrated to be clearly independent in this factor analysis. Based on the most important first configuration, it can be inferred that the results of the factor analysis relevant to importance were more about emergency situations, whereas the results of the factor analysis relevant to performance were more about daily health care.

As a result of factor analysis relevant to performance, if health and emergency sectors were considered to have similar characteristics, D and A can be understood to be singularities. Given the results of the first configuration (of factor analysis relevant to performance), D may be regarded as a health-related service/item by the young group. On the contrary, the second-factor configuration in the factor analysis relevant to performance includes services/items related to gerontification. Item categories A, M, N, and L were mentioned in the previous sections as a noticeable configuration related to the I–P value. They were mentioned as having “no distinctly common characteristics,” but it may be predicted that the older adults were the main target users. In this configuration, J was determined to be a singularity.

• Middle-aged Group:

The results of the factor analysis for the middle-aged group are shown in

Table 10. Factor analysis (middle-aged group).

Importance				Performance
	1	2	3		1	2
H	0.847	0.202	0.030	C	0.755	0.115
F	0.735	0.307	0.119	D	0.737	0.254
M	0.670	0.149	0.223	E	0.705	0.119
N	0.628	0.286	0.321	K	0.670	0.312
L	0.615	0.235	0.316	F	0.647	0.411
K	0.542	0.272	0.312	H	0.629	0.367
C	0.278	0.799	0.009	J	0.614	0.360
D	0.316	0.766	−0.020	G	0.609	0.351
E	0.115	0.647	0.350	I	0.570	0.332
I	0.213	0.561	0.325	A	0.049	0.747
J	0.331	0.504	0.377	M	0.350	0.712
G	0.411	0.430	0.353	L	0.302	0.669
A	0.212	0.157	0.824	B	0.299	0.591
B	0.229	0.112	0.803	N	0.454	0.584
Total		% variation	% cumulative	Total	% variation	% cumulative
3.400		24.286	24.286	4.458	31.842	31.842
2.788		19.913	44.199	3.057	21.838	53.680
2.186		15.618	59.817

. As all factors had a load value of 0.4 or higher, they could each be classified as a corresponding factor. For this group, each factor configuration for importance and performance had one less factor than the young group, that is, three factors for importance and two for performance.

According to the configuration in the factor analysis relevant to importance, health-related items (H, F, M, N, L K) and daily living support items (C, D, E, I, G, A, and B) were clearly independent among the middle-aged group. In the second configuration, J was distinguished by a singularity.

On the contrary, the results of the factor analysis relevant to performance were more complicated and mixed. In particular, the first-factor configuration for performance was difficult to infer. Meanwhile, the second-factor configuration for performance was almost the same as the second-factor configuration (performance: M, N, A, L, and J) for the young group, presumed to be related to gerontification. Item categories A, M, N, and L were mentioned in the previous sections as a noticeable configuration related to the I–P value and to the second-factor configuration of performance for the young group.

• Older Adult Group

The results of the factor analysis for the older adult group are shown in

Table 11. Factor analysis (older adult group).

Importance				Performance
	1	2	3		1	2
K	0.823	0.097	0.153	F	0.801	0.152
F	0.787	0.202	0.167	K	0.752	0.154
G	0.732	0.246	0.225	G	0.749	0.266
H	0.574	0.500	0.287	N	0.716	0.186
L	0.533	0.332	0.195	H	0.706	0.304
I	0.494	0.313	0.390	L	0.667	0.238
N	0.292	0.786	0.047	M	0.612	0.408
M	0.247	0.780	0.196	I	0.607	0.374
J	0.207	0.648	0.325	J	0.571	0.482
E	0.137	0.104	0.787	D	0.565	0.562
A	0.265	0.046	0.660	E	0.208	0.755
C	0.218	0.398	0.616	B	0.249	0.751
D	0.433	0.261	0.586	A	0.104	0.664
B	0.056	0.467	0.555	C	0.465	0.572
Total		% variation	% cumulative	Total	% variation	% cumulative
3.206		22.901	22.901	4.943	35.309	35.309
2.673		19.096	41.998	3.059	21.852	57.161
2.607		18.621	60.619

. As all factors had a load value of 0.4 or higher, they could each be classified as a corresponding factor. For this age group, the number of factors for importance and performance was the same as for the middle-aged group: three factors for importance and two for performance. The similarity between the two scoring categories was higher than for the other age groups.

For each item category, although the orders according to the factor values were slightly different, the overall factor classification compositions and trends were very similar. For the older adult group, factor analysis relevant to importance clearly revealed three categories (daily living, health maintenance, emergency and safety). In this factor analysis, daily living support items were clearly independent but were a third-factor configuration, not a priority consideration factor. Furthermore, G and I were distinguished from the daily living category. Instead, they were assigned to the health-related (including emergency) factor configuration. G refers to sensors that show the water temperature using color, whereas I refers to a height controller for countertops. Thus, it could be carefully inferred that older adults perceived these services as health-related, because these items provide aid for their vulnerability. Moreover, the first and second configurations in the factor analysis relevant to importance converged into the first configuration in the factor analysis relevant to performance.

4. Discussion

IPA helps understand, interpret, and amplify the “lived experiences” of the research participants and make their experiences meaningful. However, IPA has been criticized for being mostly descriptive and not sufficiently interpretative [61,62,63]. Thus, we tried to interpret unobservable latent variables that IPA analysis could not present with factor analysis.

In IPA analysis, we conceptually linked “importance” to “recognition” and “performance” to “satisfaction.” As mentioned previously, the two categories (importance and performance) began to exhibit more evident similarities with the aging of the prospective users. Therefore, as the users became older, their evaluations of importance and performance began to achieve consensus. Moreover, for all groups, H and M appeared at the top of importance evaluations, whereas C and F appeared at the top of performance evaluations. D had a high appeal for the youth and middle-aged groups, but not older adults. On the contrary, H and M had a high appeal for the middle-aged and older adult groups, but not for the young. Among all groups, the lower-ranking item categories were more similar than the higher-ranking item categories, and E, G, and L always appeared at the bottom ranks of the two scoring categories. However, A was always bottom ranked except in terms of importance for older adults. The results of the importance evaluations for the youth and middle-aged groups were similar to the results of the performance evaluation for the older adult group: among the top five items, four were common with the young group, whereas five were common with the middle-aged group.

Furthermore, except for N in the older adult group, the importance score was always higher than the performance score. This trend indicates that evaluations made based on expected future importance tend to be more generous than those based on assumed consumption. The I–P values also became smaller as the users became older. The I–P value for the young group was 0.32, that for the middle-aged group was 0.24, and that for the older adult group was 0.16. This result shows that the two categories (importance and performance) began to exhibit more evident similarities as the users became older. A, L, and M were the item categories with the highest I–P values among the young and middle-aged groups, whereas A, D, and I were the item categories with the highest I–P values among the middle-aged and older adult groups. It is also noticeable that A exhibited the largest differences between the two categories for all groups.

For a more comprehensive understanding of these data, IPA was performed. In IPA, three out of four quadrants shared common items across the age groups.

Quadrant I encompassed (C) light control, (D) bed control, (F) body composition analysis, (H) smart toilet, (I) countertop height control, and (M) fall management. These smart home-related services and items exhibit high satisfaction and market demand, which are factors to consider in their commercialization more than in their development. In particular, item categories C, D, and F demonstrated high purchase intentions among the young and middle-aged groups. On the contrary, item categories C, F, H, and M demonstrated high purchase intentions among the middle-aged and older adult groups. By comparison, Quadrant II, usually considered an important part of market evaluation, encompassed no common items for immediate improvement.

Quadrant III encompassed (A) check-in, (E) curtain control, (G) water temperature color, and (L) gait analysis, whereas quadrant IV encompassed (J) auto ventilation. Specifically, item categories A, E, G, and L were generally of low appeal to evaluators of all ages. In particular, A and E scored lowest in both importance and performance. A and E were quadrant III’s elements, indicating that these items have a low need for development and improvement, which are also the least important factors to consider in their development and commercialization. Considering the results simply and directly, it might be possible to conclude that these services are not important for all generations. However, from another perspective, further study is needed on whether the usability or appearance is insufficient or if there is another reason for these trends.

Lastly, quadrant IV encompassed J, which consistently scored low in importance but high in performance for all age groups. In South Korea, because of the recent universalization of air purifiers, J as a service seems to be less appealing. On the contrary, in the living lab, A, B, C, and J were designed to optimize energy consumption; however, this may not have made a significant difference to the experimental experience of the evaluators.

The fact that IPA focuses on cognition is problematic and limiting to our understanding because empirical research seeks to understand lived experiences but does not explain why they occur [64]. An authentic research inquiry seeking to understand the experiences of its participants will also seek to explore the conditions that trigger the association of experiences relating to past events, histories, or sociocultural domains. However, this study had limitations, and at the simplest level, the weaknesses of the study were based on factors such as duration of use. Thus, we adapted the factor analysis to build up the methodology firmly.

The next phase, comparing the factor analysis results among the age groups (Figure 7), was an attempt to infer the cause of the difference between the two scoring categories. Across all groups, the factor analysis relevant to importance firmly revealed the significance of health-related items. Daily living support items were ranked as a less important factor. However, the results of the performance analysis revealed mixed configurations of items related to daily living support, health, and emergencies, especially for the middle-aged and older adult groups. Overall, the factor configuration of the performance analysis showed a mixture of various categories compared to the item composition of the importance analysis. In other words, it is possible to assume that the heterogeneity of the factor configuration as a result of the analysis of performance was the result of considering actual needs when the prospective user presupposed purchase.

It was also found that the second-factor configuration (performance) for the young group was almost the same as the second-factor configuration for the middle-aged group, whereas these factors converged into a first-factor configuration in the performance evaluation by the older adult group. In the previous chapter, common item categories A, M, N, and L were noted as prominent constructs in relation to gerontification. Comparing the factor analysis of “importance and performance”, it can be inferred that notions of gerontification and needs of older adults influenced the judgment of prospective users.

5. Conclusions

IoT technologies and products have been rapidly changing our daily lives, with some of these technologies already permeating the houses of today. However, even though these products and services are being seen far more frequently than before, their use has not yet become successfully pervasive throughout our society. In a previous study, it was determined that there are many reasons that can affect surveys regarding user preference for smart technologies, and that people’s conceptual evaluations and practical choices tend to be different [9]. Among research efforts on future housing, many studies were based on this type of conceptual evaluation, determined through surveys [65,66,67,68]. Therefore, in this study, there was an attempt to evaluate a variety of smart home services, to better understand the recognition by and satisfaction of a user in a more empirical manner. Subsequently, “importance” was conceptually linked to “recognition” by the people and “performance” to “satisfaction” of the people. This was because evaluators marked the importance scores based on the recognition that the item will be important (for future housing) and marked the performance score based on whether their satisfaction was sufficient to motivate them into purchasing the item.

According to the statistical analysis, the two scoring categories tended to have a low similarity among the young group but began to exhibit evident similarities with the aging of the users. In other words, the importance and performance scores in the older adult group were more similar than in the other groups. On the contrary, the results of the importance evaluations for the young and middle-aged groups were analogous to the result for the older adult group, whereas those of the performance evaluations were not. Furthermore, the lower-ranking items for the I–P evaluations were consistently common among all age groups; by contrast, the upper-ranking items tended to differ. There were also clear similarities regarding which items did not appeal to any of the groups.

For further understanding, a factor analysis was performed. As mentioned previously, the scores for the two categories began to exhibit evident similarities with the aging of the users. Factor analysis was used to gain an understanding of the different structures among the factors underlying the measured items. It is noteworthy that the second-factor configuration (performance) for the young group was almost the same as the second-factor configuration for the middle-aged group. Furthermore, in relation to the aforementioned result, the third-factor configurations (performance) for all visitors offered a hint about the feature. If there were no similarities in the configurations (performance) for the two groups, we might think that the second-factor configuration was approximately about emergency response. However, with the results for the middle-aged and older adult groups, the feature of the configuration was deduced as relevant to older adults: gerontification and, more specifically, vulnerability. This is certainly related to health care or emergency response, based on the close intertwinement with the physically debilitating conditions that tend to be present in older adults. Such physically debilitating conditions cannot be a significant factor in performance evaluations (i.e., satisfaction is sufficient to motivate the purchase of the item) in the young and middle-aged groups. However, that factor converged into an underlying cause for first-factor configuration in performance evaluation by the older adult group. By comparison, among the young and middle-aged groups, services and items related to daily health maintenance and management tended to be more appealing. More generally, it is reasonable to infer that whereas health-related services and items are considered “important” by all groups, user vulnerability affects the priority of these services and items in terms of “performance.”

To sum up, for the design of future smart homes suitable for an aging society, the development of services/items that can constantly check the health status of the resident should be given priority. For smart homes with older residents, the development of services/items that can support them in their vulnerability should be emphasized. Eventually, more successful developments (i.e., that motivate the purchase of the item) should prioritize services and items that can actively support health-related vulnerabilities caused by aging. For a relatively young population, items with automatic detection and response functions in daily life are attractive. However, for this favorable impression to lead to a purchase, it is preferable that a health factor is included. In South Korea, most active health services have not yet been developed in different aspects because of restrictions related to medical services. To develop smart homes that are more practical and suitable for user needs in the future, follow-up studies should be conducted, and measures to overcome these limitations should be proposed.

Smart homes are an advancing wave of technological development whose success depends on a coalescence between the visions of technology developers for enhanced functionality and energy management, and the needs and demands of households in the complex places that are homes. Yet there is a wide and growing recognition of the need to develop a better picture of who users are and how they might use smart homes [69,70,71]. This study discusses evaluations by prospective users of smart home technology and products, to enhance the existing understanding of service and item needs for the optimal commercialization of smart homes for users. Smart home testing procedures typically attempt to capture the “user experience” in a lab or single-occupancy apartment and are often criticized for missing the intricacies of home life, where practices are shared and negotiated between residents and visitors with different priorities [6,69,70]. One cannot help but point out that this study has the same limitation. Experience in the smart home lab would be completely different from real life and it would be difficult to reflect on its complexity. The findings reported here cannot, therefore, be simply generalized to all types of smart home service and items contexts.

However, perhaps the main significance of this study is that it extends our understanding of “user evaluation vs. choice” and our understanding of different demands for smart home performance in each age group. Moreover, by comparing the results of factor analysis of each age group, it was possible to infer the users’ concealed considerations. It may even encourage ideas about properly complex but practical responses to the wicked issues posed by our reliance on user evaluation. Careful development of services and items that satisfy these considerations and their outcomes will be a necessary part of designing and carrying out smart home performance.