which means they think of the information they need in their databases to process an application and ask applicants for this information using administrative language. The issue is applicants think about forms in very different terms. They want to study in a given program and the form is a means of achieving this goal. Most studies tend to address problems with forms by focusing on the technological and aesthetic attributes of a form. However, having a well laid out and interactive form does not address the problem if applicants do not understand the application process, if the form is not structured properly and if questions are poorly constructed.

What started out as a vague idea about working with forms during my first year of graduate studies became a master's thesis and I have many people to thank for being able to accomplish that. There is my thesis director, Isabelle Sperano, who was also my tutor and internship supervisor. She has shown me there are many paths to user experience and provided invaluable insights about writing this thesis. Thank you for sharing your expertise and for your availability.

There is Caroline Jarrett, a forms consultant based in the United Kingdom, who took time to discuss my thesis and made suggestions on how to conduct the user tests.

There are the students and recent graduates who participated in the user tests. It would be hard to make any progress in the field of user experience if the users, the people whose experiences we seek to improve, did not provide feedback.

I also thank the teaching body of the master Architecture de l'information at École normale supérieure de Lyon who, each in their own way, have contributed pieces of knowledge that have enriched this thesis.

Last but not least, I am grateful to my family and friends for their support and words of encouragement.

Starting with an "outside in" perspective concerned with understanding how applicants view the application process and think of the information they need to achieve their goal may help address this potential gap. The aim of this thesis is to examine five graduate school application forms from institutions in France to identify trends, potential issues in the use of forms and use this information to make recommendations. Another goal is to understand the users' application experience to make recommendations. To achieve these goals, a corpus analysis was conducted to identify trends in application forms and potential hindrances. Then, interviews were conducted with students and recent graduates to understand their past application experiences. The interviews were followed by tests with the same individuals to observe how they used application forms, the difficulties they encountered and the good practices they highlighted.

The findings from this thesis can contribute to information architecture in many ways. They can help institutions identify information gaps in the application process for graduate studies. They can encourage user experience professionals to consider other dimensions of forms such as the structural patterns of a form. Lastly, addressing issues with forms from the perspective of information design can raise awareness of how better forms design can improve communication between organizations and users, and service delivery.

Glossary Admission

Admission (admission) is the process undertaken by a prospective student seeking entry into a study program.

Application Form

The application form (formulaire de candidature) is the main document prospective students fill in and submit to apply for entry into a study program.

Form

A form is an interface used to collect information and visually distinguishable by its fields (or boxes) in which users input data. The focus will be on forms users fill directly in a browser.

Information Design

Information design is an area of design concerned with making complex information clear so users can use it to perform tasks.

Macrostructure

The macrostructure designates three aspects of the form: structure, content hierarchy and content organization.

Mental Model

A mental model is what a user believes about a system and refers to what users know and think they know about forms and the way they are structured.

Microstructure

The microstructure designates three aspects of a section: how fields are arranged, the order in which they appear and how questions are worded.

Service Design

Service design is an approach focusing on the global experience of students to support and improve service delivery.

Suprastructure

The suprastructure designates everything outside the form and includes the institutional website, how users access the form and the application portal.

User Experience (or Users' Experience)

User experience designates how users perceive their experience using a given form.

Notably, what happens when they read a question, try to understand it, find the answer and judge the suitability of their answer before writing it down on the form.

Introduction

Forms are part of many online transactions such as shopping, joining a social networking site or accessing web-based productivity tools [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF]. Technological advancements have added interactivity to forms and improved their accessibility, but have not made them necessarily easier to use. An indication of this is the problems with forms today are similar, if not the same as those observed 10 years ago [START_REF] Barnett | Designing Useable Forms: Success Guaranteed[END_REF][START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF][START_REF] Schwesinger | Form follow user follows form. From designing forms to designing services: a tutorial[END_REF] and those identified in the middle of the 1970's when scientific research on forms started [START_REF] Barnett | Designing Useable Forms: Success Guaranteed[END_REF]. Despite the lessons learned from these studies, many organizations continue to underestimate the impacts of poorly designed forms on business performance [START_REF] Barnett | Designing Useable Forms: Success Guaranteed[END_REF].

In e-commerce for example, problems with forms have been found to increase the rate of customers abandoning their cart at checkout resulting in revenue loss and decreased performance [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF]. In the public sector, studies have shown difficulties with forms influence how citizens perceive the modernization and accessibility of their government [START_REF] Jansen | How research can lead to better government forms[END_REF]. In addition, similar research has demonstrated complex forms tend to generate a high error rate resulting in staff spending considerable time correcting errors that could have been avoided [START_REF] Barnett | Designing Useable Forms: Success Guaranteed[END_REF]. Lastly, poorly designed forms can negatively impact data quality, affect the efficiency of business processes and ultimately, the performance and services of an organization such as a university [START_REF] Kim | Alexander and the Terrible, No Good, Very Bad Web Form[END_REF].

Forms play an essential role in the transactions between businesses and their customers [START_REF] Schwesinger | Form follow user follows form. From designing forms to designing services: a tutorial[END_REF] and between prospective students and higher education institutions. Application forms can simplify the user's experience and improve an institutions' service delivery if they are designed carefully. Conversely, they can complicate the user's experience and hinder an institution's efficiency if they are not designed to support users appropriately. This is particularly important when considering the investments institutions make to attract prospective students. A survey conducted in 2017 by the French association of communication professionals in higher education (Association des responsables de communication de l'enseignement supérieur), found universities had, on average, a budget of 199 000 € for communication activities. A significant portion of that budget was dedicated to student recruitment and for 40% of respondents, this activity represented at least 50% of the allocated budget. Application forms have a strategic role as intermediaries in the conversation between institutions and prospective students even though they may be an under-noticed area of the recruitment process. It is therefore critical to understand applicants' experience and examine forms to uncover potential hindrances and good practices that can advance knowledge on forms in the context of the application process for graduate studies.

Research Problem

Research Gaps in Forms Design

Forms are everywhere but research gaps remain in higher education. One of the reasons for this is most studies focus on forms used in e-commerce (Wroblewski, 2008, Jarrett and[START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] and the public sector (Barnett, 2007, Jarrett and[START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF]. Studies on forms in higher education mostly consist in best practices and benchmarks (Mackin, n.d., Trasher, 2014, Tank, 2016, Ricoh, 2017, Larsens, 2013). Another reason is studies on forms in higher education tend to be based on industry best practices and case studies rather than academic research. The studies focus on a specific form, providing results less generalizable than those from a large sample of forms would. In addition, the studies usually concern application forms used in American institutions which may be subject to different requirements than application forms in other countries. Lastly, these studies tend to articulate problems with forms as technology and interaction design issues and overlook content, structure and wording in forms [START_REF] Rees | Caroline Jarrett discusses forms, surveys and the need to be brave[END_REF]. This could be the reason organizations tend to focus on the aesthetic design of forms as opposed to forming good questions and structuring content. However, businesses underestimate the impact of poorly designed forms on brand perception and user experience.

Impact of Poorly Designed Forms in Higher Education

Standing between a prospective student's goal to enter a program of study and an institution's recruitment goal is an application form. The form is a step in the admission process and today, many applicants expect this process to be digitized (Ricoh, 2017) like other transactions they do. In the United States for example, [START_REF] Frazier | An Admission Process Transformed with Technology[END_REF] observed students would not even consider an institution if the admission process was not available online. Along the same lines, [START_REF] Lindbeck | Using Technology in Undergraduate Admission: A Student Perspective[END_REF] found 88 percent of prospective college students would be disappointed or would consider eliminating an institution from their choices if the website did not meet their expectations. Lastly, [START_REF] Sherwin | University Websites: Top 10 Design Guidelines[END_REF] points out prospective students often complain about not being able to find information on the application process on the institutional website. These results show how technology can influence prospective students' perceptions of an institution, reflect on the credibility of an institution and impact the decision of a prospective student to attend a given institution.

In the same way, a poorly designed application form can undermine the perception prospective students have of an institution. In its 2015 form conversion report, Formstack, a data management company, reviewed the forms of 1 192 clients in higher education and found the rate at which users completed forms in this sector averaged 10%. The company also found users were less likely to complete forms asking too much or irrelevant information. A similar study by Formisimo, a company specializing in form analytics, examined university enquiry request forms and found longer forms tended to result in greater abandonment rates (Mackin, n.d.). These results suggest prospective students can be put off by overly long application forms.

In addition, if prospective students do not understand the purpose and value of the information an institution is requesting, this can create a negative application experience, tarnish the institution's image and ultimately, impact the decision to attend an institution.

Impact of Poorly Designed Forms on the Admission Experience

The project of choosing a university can be exhilarating but filling forms can be a burdensome activity [START_REF] Barnett | Designing Useable Forms: Success Guaranteed[END_REF], and for prospective students, it can also be a stressful experience. For instance, when an application form contains questions unrelated to the admission process, prospective student can become confused or frustrated [START_REF] Kim | 7 Common Web Form Design Mistakes to Avoid[END_REF], Formstack, 2015) if they do not understand the value of the information for what they are trying to accomplish. Furthermore, it can be irritating for prospective students to consistently backtrack to previous steps when the application form doesn't follow a logical flow. This back and forth makes the application experience chaotic and can diminish the level of confidence prospective students have in an institution [START_REF] Kim | Alexander and the Terrible, No Good, Very Bad Web Form[END_REF]. Lastly, application forms can appear intimidating and overwhelming (Trasher, 2014) when they contain terms and concepts prospective students are unfamiliar with [START_REF] Sherwin | University Websites: Top 10 Design Guidelines[END_REF]. Prospective students may feel they will be unable to fit in or conversely, feel the institution is out of touch with its audience. These problems show poorly designed forms can increase the complexity of filling in an application form for prospective students [START_REF] Kim | 7 Common Web Form Design Mistakes to Avoid[END_REF]. They also show these problems can create a psychological and emotional burden on prospective students.

Research Goals and Questions

To address the gaps in the design of application forms in higher education, this thesis will identify trends in the design of graduate studies application forms in France, examine users' past application experiences and gain insights on the main issues with the application process and the use of graduate studies application forms. In doing so, it will attempt to address the following questions:

-What are users' expectations and concerns about the admission process? -How is information organized in a graduate studies application form? -What issues do users encounter using a graduate studies application form?

The goal of these questions is to understand users' application experiences, the structural patterns in application forms for admission to graduate studies, the extent to which the forms match users' expectations and potential issues with the use of these forms. The results can provide a better understanding of the issues prospective students face with application forms and how to improve user experience and service delivery. To start, however, it is important to understand the context in which graduate studies application forms are used which is what the next section will describe.

Admission Process in Higher Education

The admission process in higher education in France can be divided in three stages:

-application (candidature), where prospective students prepare and submit their application form and supporting documents -registration (inscription administrative), where admitted students pay tuition fees and obtain their student card -enrollment (inscription pédagogique), where students select the courses they intend to follow during the academic year The admission process for undergraduate studies varies from the one for graduate level studies. Applications to the former are submitted through the online platform ParcourSup (https://www.parcoursup.fr/) while applications to the latter are submitted to each institution directly, except for non-European union nationals from specific countries who must apply through the agency Campus France.

Marketing Approach to the Admission Process

Much of the traditional approach to the admission process in higher education has drawn from marketing practices. Consulting agency Noel-Levitz and the Association for Graduate Enrollment Management (Coen, 2012) for example, classify the admission process in seven key stages ranging from prospect to enroll.

Similarly, [START_REF] Rollot | Livre blanc: La communication dans les établissements d'enseignement supérieur 2015/16[END_REF] white paper on communication practices and approaches in French higher education institutions shows student recruitment is perceived in terms of "digital marketing", "strategic communication" and "corporate identity". Marketing is concerned with metrics such as the number of applications and admitted students. However, this approach can fail to see the broader context in which these activities take place and omit the experiences of prospective students. As a result, it can overlook what happens during the application stage. One reason could be that marketing departments are rarely involved in the application stage. [START_REF] Rollot | Livre blanc: La communication dans les établissements d'enseignement supérieur 2015/16[END_REF] found only 20% of communication directors were integrated to their institution's admission or welcome services. This speaks to the importance of having a view that sees the application stage as part of a bigger process.

Integrated Approach to the Admission Process Through Service Design

The marketing approach focuses on metrics whereas other studies [START_REF] Baranova | Service Design in Higher and Further Education[END_REF]Mutton, 2010, Felix, 2011) suggest taking a holistic approach to the admission process based on service design. This approach focuses on the experience of students as a whole rather than on each stage individually to support and improve service delivery. The British group Supporting Professionalism in Admissions (SPA) (2017) for example, divides the admission process in four stages (Figure 2):

-pre-application, where prospective students demonstrate an interest for higher education studies -application, where prospective students apply for admission in a program -post-application, where institutions review and assess applications and notify applicants they have been admitted -transition, which includes all activities following admission confirmation The SPA's flowchart illustrates the admission process within the student journey (Figure 2). It does not discuss the particularities of pre-application and graduation, but it presents the admission process as part of a broader process. Approaching the application stage from the view of service design can help shift the focus from the institution to the applicant. This is important to be able to understand prospective students' experience with the application form, how it shapes their perception of the process's quality, and their overall experience. Contrarily to marketing approaches, service design focuses on human interactions at key stages of the conversation between an organization and a user [START_REF] Felix | [END_REF] and is concerned with what makes a service more effective for those using it. In this thesis, it can help understand applicants' expectations and concerns and how they are addressed throughout the admission process. After describing the context, attention is given to understanding what forms are and the purpose they serve in the next section.

Concepts and Approaches to Forms Design

This section provides a review of the existing literature and the main concepts related to forms design. The first part presents the different types of online forms. The second describes the functions of forms. The third explains how forms have been compared to a conversation. The fourth provides an overview of guiding principles taken from user research to improve the design of forms.

Types of Online Forms

Forms are interfaces available in different formats such as paper or the web [START_REF] Enders | Designing UX: forms[END_REF]. The main visual indication that a document, or part of a document, is a form is the presence of fields or as [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] put it the "boxes you can type into". What further distinguishes a web form is that it is contained in a web page or browser [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF]Gaffney, 2009, Enders, 2016). [START_REF] Killam | Forms Management: What Forms Managers Think About[END_REF] describes five types of online forms which are synthesized here starting with the least integrated:

-print-on-demand (POD), which users download and send manually -fill-and-print (FP), which users fill online, print and send manually -intelligent electronic forms (IEF), which users fill in their browser, print and send manually -entreprise-enabled (EE), which are connected to the organization's databases and use basic features such as email connections -complete business applications (CBA), which forwards users' information to the different parts of the organization for processing after their submission This thesis focuses on CBA forms which few institutions in France use. In the Université de Lyon consortium for example, 4 institutions out of 25 have an online procedure and in the Université Paris-Saclay consortium, 3 institutions out of 13. This is a small number but still interesting to examine how institutions handle these forms.

Organizational and Social Functions of Forms

Forms can have different functions. [START_REF] Jansen | How research can lead to better government forms[END_REF] for example, find they have three organizational functions in government:

-enforcing regulations -transferring data from citizens to the government -fostering good public relations [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF], on the other hand, finds online forms are mostly used for:

-e-commerce (checkout forms) -community building (registration and contact forms) -productivity (login forms) Schwesinger ( 2017) considers forms' functions based on the setting:

-managing transactions between customers and businesses -managing relationships between citizens and government -managing relationships between employers and employees -organizing and structuring processes between businesses Likewise, university application forms can have similar functions:

registering applicants' request to join a study program transferring applicants' personal information to universities processing applicants' application file to assess their eligibility and suitability -managing communications with applicants A graduate studies application form can have an organizational and a social role. However, an institution can miss the mark when it overlooks the role forms play in dealing with applicants during the admission process and in speaking about itself to the outside world [START_REF] Schwesinger | Form follow user follows form. From designing forms to designing services: a tutorial[END_REF]. If they are to guide interactions with applicants, then it is important forms foster good public relations with users and be coherent with the image and the brand of the institution. For this, forms have to be better integrated in the communication strategy and the steps a student go through during the admission process.

Forms as a Type of Information-Seeking Dialogue

Many authors view forms as a means of communication and compare them to a conversation. [START_REF] Sless | Designing and evaluating forms in large organizations[END_REF] for example, takes a constructivist perspective and suggests forms are a means of enabling a conversation, building a relationship between an organization and users. From this view, parties construct their understanding of the world and of others, through communications, which in this instance are taking place by means of a form. [START_REF] Jansen | How research can lead to better government forms[END_REF] describe forms as a one-sided, asynchronous and asymmetrical dialogue that tends to involve an anonymous interlocutor inside an organization and an identifiable user. The organization decides the topics, questions and format of the answers and to achieve their goal, users have to accept the terms of the conversation and answer the questions in a format which may not always suit the reality they wish to convey. In the same line of thoughts, [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] describe forms as a sort of remote conversation in which there is no physical interlocutor to obtain clarifications from. Maier (2014) examines the processes, protocols and the platforms involved in managing forms and describes it as a unidirectional conversation from users to an organization. [START_REF] Schwesinger | Form follow user follows form. From designing forms to designing services: a tutorial[END_REF], on the other hand, examines the setting in which forms are used and notices how they are expected to replace personal communication with an organization while at the same time, initiating a dialogue between an organization and users.

Authors often use face-to-face conversation as a metaphor for the components of a form. [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] seem to be the first to use this analogy to describe the sequence of questions and answers in a form. [START_REF] Enders | Designing UX: forms[END_REF] uses the same analogy to explain there is a time and context to ask questions and a way of wording questions to obtain users' co-operation. Forms should not feel like an interrogation where the organization imposes the terms of the conversation on users. It should resemble an interview where there is an exchange of information between the organization and users.

In the same way, graduate studies application forms are a device to initiate a conversation between an institution and prospective students. Institutions provide a form which applicants fill in and send back. Institutions then use the form to assess applicants' eligibility and notify them of their decision and next steps such as registration. Understanding forms as a conversation, shifts the attention away from the organization to the user [START_REF] Schwesinger | Form follow user follows form. From designing forms to designing services: a tutorial[END_REF]) so consideration can be given to how applicants understand questions in the form, how this interpretation shapes their application experience and what are the obstacles to comprehension.

Principles to Improve User Experience With Forms

A number of publications have suggested ways of improving forms and user experience. Following are a few principles taken from information design, which is concerned with making complex information clear so users can use it to perform tasks, on form structure, content and wording.

Structure

-questions are organized in themes [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF][START_REF] Enders | Designing UX: forms[END_REF] -themes and questions follow a logical sequence, from the general to the particular [START_REF] Enders | Designing UX: forms[END_REF] -section headings match the content [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF] -long forms start with an explanatory introduction [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF] Content questions unrelated to users' goals are avoided [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] -when the reason for a question is not straightforward, its purpose is explained [START_REF] Enders | Designing UX: forms[END_REF] Wording -concepts and words are easy to understand by all users [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF]Gaffney, 2009, Enders, 2016) -the tone is adapted to the context and audience [START_REF] Schwesinger | The Form Book: Creating Forms for Printed and Online use[END_REF] Framing forms as a conversation, relegates technology and aesthetics to a secondary position. If forms are to effectively initiate a dialogue between institutions and prospective students, it is essential to understand forms as applicants understand them. The next section will describe the methodology put in place to be able to achieve this.

Methodology

This section describes the methods used to answer the research questions and achieve the research goals. First, a corpus analysis was conducted on five forms to understand how graduate studies application forms were organized. Then, interviews were conducted with users to understand what they believe about these forms, followed by user tests with the same individuals to identify pain points.

Scope of Examined Graduate Studies Application Forms

The scope of this research was limited to graduate school application forms used by member institutions of the Université de Lyon and Université Paris-Saclay consortiums to compare application forms of institutions located in the capital with those of institutions located other cities with a considerable student population. Focusing on graduate studies application forms meant users came with prior experiences which they gained from applying for undergraduate studies. It was an opportunity to understand how users' past application experiences influenced their mental model, which is what a user believes about a system, and explore expectations, concerns and perceptions. Second, it excluded from the pool of potential users non-European Union nationals who must apply for graduate studies through Campus France. Lastly, focusing on institutions using an online admission procedure limited the choice of application forms that could be analyzed since few institutions seemed to use this type of procedure in France.

Corpus Analysis

The corpus analysis consisted in comparing the graduate studies application forms of five institutions. The goal was to identify components and themes to understand structural patterns and compare them to what users believe about forms to see if it matched their expectations and addressed their concerns.

Corpus Composition

The study examined the online application forms of five institutions (Table 1). Comparing the application forms of different types of institutions helped understand if specific types of institutions shared common points and having forms from institutions in different locations helped understand if identified trends could be specific to a region in France.

Corpus Inclusion Criteria

Institutions were chosen from the Université de Lyon and Université Paris-Saclay's list of member institutions and had to meet the following criteria:

-the main language of teaching is French -the institution offers graduate studies programs -the admission procedure is completely online -the application forms are accessible in February and March

It was noticed that several institutions use an admission software eCandidat, developed by the consortium ESUP-Portail (https://www.esup-portail.org/). In this instance, only one institution was kept in the corpus (Lyon 1) and the others excluded because the goal is to identify common design patterns in different application forms.

Access to the Graduate Studies Application Forms

Application forms were accessed by signing up for accounts as real applicants would do. On the suggestion of a forms consultant (C. Jarrett, personal communication, December 18, 2017), improbable information was purposely provided in the sign up form and the application form so institutional staff know the application is fictitious (e.g. applicant name: Thisa Test and address: 123 Test).

Analysis of the Graduate Studies Application Forms

Two grids were used to analyze the application forms. The first grid was based from Wroblewski's analysis grid for checkout forms ( 2008) and maps out the content of each form (Figure 3). It was created using a spreadsheet in which the organization of content and fields in a page or screen were represented. The second grid contained a list of principles to improve user experience with forms on aspects such as navigation, content, structure and wording (Figure 4) (See annex 1 on p. 50 for the full grid). These principles were taken from previous studies on forms [START_REF] Wroblewski | Web form design: filling in the blanks[END_REF][START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF][START_REF] Enders | Designing UX: forms[END_REF]. A first version of the grid was tested with a form and where needed, adjustments were made to adapt the principles to the context. When needed context-specific principles that seemed important were added. Each application was then reviewed using the grid and notes were taken to identify issues and trends. The goal was to compare and analyze the content organization of the application forms and identify potential hindrances in advance of user tests.

User Interviews

To understand users' application experiences, interviews were conducted with individuals who had applied for undergraduate or graduate studies in France. The goal was to understand their expectations, concerns and overall perception of their application experience. A second goal was to identify potential gaps in the admission service chain and use this information to suggest improvements.

Organization and Conduct of User Interviews

Users were recruited through social networks and personal connections and had to meet the following criteria:

-speak and understand French -be current students or recent graduates (less than a year) -have experience applying for undergraduate or graduates studies in France -if they were non-European nationals, be allowed to apply directly to institutions or already be studying in France -accept to participate in a user test after the interview and agree to the recording of their comments for later analysis It was not possible to work with real applicants despite reaching out to student associations in each of the five institutions to disseminate information about this research and other institutions in Lyon. The pool of eligible participants was therefore extended to current students or recent graduates. The decision to include recent graduates seemed appropriate since it is not uncommon in France to pursue a second master's degree. The interviews usually lasted between 30 and 60 minutes and were conducted in-person or remotely. Interviews explored participants' past application experiences and how they thought the form would be organized.

Analysis of Interview Data

The interviews were transcribed and analyzed using the affinity diagram method to bring out the main themes emerging on users' expectations, concerns and perceptions about the form and the application process, identify gaps and make recommendations.

User Tests

User tests were conducted with the same participants from the user interviews and consisted in observing how they reacted when using the form to identify areas where they encountered difficulties using the application form.

Organization and Conduct of User Tests

Tests were conducted with two participants for each form. [START_REF] Jarrett | Forms that work: Designing Web forms for usability[END_REF] suggest having three users as this usually helps catch most problems in a form but as explained previously, recruitment efforts were not successful. Tests were moderated using the think aloud protocol to understand what participants thought as they were going through the form (U.S. Dept. of Health and Human Services, 2014). Participants were asked to fill in the form as far they could without submitting it.

Analysis of Test Data

Recordings were transcribed and analyzed. Qualitative results were generated using participants' comments. These comments were analyzed to identify recurring themes inside each form and across different forms and integrated in the findings to cross the findings from the corpus analysis with participants' experiences with the forms. The following sections summarize the main findings uncovered with the methodology starting with users' application experiences.

Findings From User Interviews

What follows in this chapter are findings from the interviews conducted to understand users' application experiences. The first section looks at users' expectations. The second, looks at their main concerns and the third, their overall perceptions. The last two parts of this section discuss the impact of users' expectations on their perceptions and suggest recommendations.

6.1 Users' Past Experiences Applying for Undergraduate and Graduate Studies

Users' Expectations About the Application Process

To understand users' past application experiences, one of the questions during the interview focused on expectations they had about the application process and the application form. Users' expectations can influence their perception of an institution so it was important to be aware of what they had hoped to see, hear and feel and see if it had been addressed.

Users had two expectations. Performance expectations such as having a "simple application experience" (P8) or a "faster response" (P2, P6) (Figure 5). Then, there were interpersonal expectations which a user compared to the relationship between businesses and clients saying:

We want the customer service, we are a little bit like customers because we pay, to be good service. I want people to be polite, show they are interested in me, show they are interested in getting good candidates. (P9)

Figure 5 -Users' Expectations During the Application Process

Performance expectations refer to the quality of the process and interpersonal expectations to the relationship between the institution and the applicants. This seemed to indicate users placed high importance on usability and on elements providing them a positive emotional experience during the application process.

Users' Concerns About the Application Process

In addition to their expectations, users were asked if they had concerns about their application or the admission process. Three concerns were identified (Figure 6): wrongdoings from the user, wrongdoings from third-parties and concerns that had more to do with insecurities. The first type of concern users expressed was of making mistakes during the application phase. They were conscious an incorrect or incomplete application could result in delays in the processing of their application or rejection and this made them anxious. There were also worries, as this user explains, of misunderstanding instructions when applying to study in a different language:

"If I'm applying to an institution in a foreign country, sometimes I am worried, especially if it's in a foreign language. Most of the time, I'm not quite sure of the things they are asking, if I am submitting the correct documents that they want me to submit. " (P10)

Misunderstanding can be due to insufficient knowledge of the language, but also unfamiliarity with the education system or with the institution's jargon. Therefore, the concern raised could be cultural and linguistic.

The second type of concern users expressed was the mishandling of their application by third-parties, notably institutional staff. Once they had submitted their application, they feared their file would not be processed appropriately, especially when the admission process was not online as this user shared:

[...] especially when the application process is paper-based you fear the staff will lose your documents or hope they have received your envelope. (P9)

Lastly, the third type of concern had to with insecurities users had about themselves. The fear of not being admitted was a recurring response along with the possibility that one's profile may not be suitable as this user commented:

I had not received the acknowledgment receipt so I was a little stressed and telling myself, 'Will I be able to do medical school or not? Is it that they don't want me, you see? Is the problem that I'm not good enough? They looked at my application and it's not good? (P6)

One user didn't have any concerns because, in his view, the application process was "just a formality" (P7) since there was no selection. On the whole, however, what users seemed to express was the experience of a persistent state of anxiety.

Users' Perceptions of their Application Experiences

To understand the connection between expectations and satisfaction, users were asked to share their general impression of their application experience. The responses reveal satisfaction depended on two factors: the type of application process and the level of assistance.

When admission procedures relied on a national platform centralizing all applications, users generally had relatively good perceptions. This was the case for users who had applied using Admission Post-Bac (APB) in France. It is the former national application portal for students in the final year of high school, first-year university students under 26 years old changing programs and some international students (M.E.S.R.I., 2017). Those who used it agreed the logic behind the process was obscure, but they liked the one-stop shop system. For one thing, it reassured them they were in the right place as this user pointed out: Second, APB being used nationally, users obtained a lot of assistance. They recalled receiving information from their teachers, parents, online and in some cases, had participated in sessions organized at their school. The application form was pre-filled by the school with their identity information and all they had to do was enter their choice of programs and ranked them in order of preference making the process relatively easy.

On the other hand, users who had applied directly to institutions usually had mixed reviews of their application experience. They found the workings of the admission process were not always clear. The institutional website didn't contain the information they were looking for. In those instances, users relied on their network. They sought help from teachers, friends or family members who had been admitted to the same institution in previous years. However, they never contacted the institution they were applying to. Sometimes, it was fear as this user shares: For another user, on the contrary, struggling was a normal part of the experience:

I didn't ask members of the institution because I knew it was also part of the game. They make it complicated to discourage people who are not really motivated so I asked my preparatory class teachers who had heard of the experiences of other students who had done the same thing. (P4)

Users' views of their past application experiences depended a lot on the information and assistance they received (Figure 7). When they applied through APB, they had information and support. However, when they applied directly to institutions, this was much less the case.

Impact of Expectations and Concerns on Perceptions

From the results, it seems there is a connection between users' expectations and their level of satisfaction, with the fulfilling of the former greatly influencing the latter. Users' expectations centred on a process that was easy to use and provided timely feedback throughout the process. These expectations went hand in hand with users' concerns about the application process. Whether real or perceived problems, users seemed to doubt their ability to fill in the application correctly, distrust the institutional staff's competencies to process their file appropriately or lacked confidence in themselves. This seemed to indicate there were often conversation gaps in the process. Users hoped to be reassured about their application but generally received no response from the institution which hindered the perceived quality of the process and influenced how users perceived their experience. This was explicit in a user's response where she compared her experience applying for undergraduate studies with APB and her experience applying for graduate studies: When expectations about information and communication were met, which was relatively the case with APB, the experience was more positive due to there being a lot of information and communication. It addressed some of the users' concerns such as having access to information on the application process and the need for assistance. On the contrary, the application process carried out outside APB or directly with institutions often did not address these concerns and resulted in much less satisfactory experiences. From these results, it seems clear information and communication are essential elements to consider to improve the application experience.

Recommendations to Address Users' Information and Communication Needs

The expectations and concerns of users indicated there were information and communication needs at every step of the application process. When considering how best to address these needs, it is helpful to remember principles of a good dialogue such as turn taking. Institutions have questions for prospective students and prospective students have questions which they also want the institution to answer and clarify. The ability to communicate, answer these questions and provide feedback is therefore crucial to meet users' expectations and address their concerns.

Being aware of these expectations and concerns can also help determine the type of information and communication needed to reassure applicants (Figure 8). In the preapplication stage, this can mean more information on the program of study and the application process so applicants know what to expect. In the application stage when applicants fill in the form, it can be ensuring the system provides applicants feedback on their progress and informs them when they have entered information incorrectly.

When applicants have submitted their application, it can take the form of an acknowledgment receipt to confirm the application was received and a notification every time the application file passes a different processing stage. These recommendations could ensure users have the information they need at each stage of the application process and reassure them. These suggestions may not take away the anxiety of not being admitted most applicants feel, which is to be expected and normal up to a certain extent, but they can alleviate the concerns resulting from an asymmetry of information. After summarizing the findings from the user interviews which gave some insights on the expectations, concerns and perceptions of users and a part of their mental model, the next section summarizes the findings from the corpus analysis and the issues identified including difficulties encountered during user tests.

Expectations

Concerns

Improvement Recommendations

Findings of the Corpus Analysis and User Tests

What follows in this chapter are findings from the corpus analysis and user tests with the application forms of five programs (Table 2) to identify trends, understand their structural patterns and identify issues with the use of these forms. The findings in this section are divided in three parts focusing on a layer of forms: suprastructure, macrostructure and microstructure (Figure 9). Each section starts by describing the interfaces and the trends identified with the corpus analysis and presents the main issues identified with the analysis and user tests. This section presents the findings of the analysis of the suprastructure. The term "suprastructure" is inspired from the work of [START_REF] Kavanagh | Anatomie et fonctionnement du schéma PANATER en architecture d'information Web[END_REF] and Sperano (2017). It refers to everything before the form starts. It includes the information on the application process on the institutional website, how people get to the application portal and the sign up form which are the parts that will be focused on. These elements are outside the form but they can significantly influence user experience. Hence, when examining forms it is important to consider not only the form itself but also everything around the form.

Information Available on the Application Process

The first section of the analysis of the suprastructure will look at the program page to examine the information on available on the application process. This seems logical considering findings on users' past application experiences highlighted the importance of information on the application process before applying.

The program pages of each provided varying levels of information about the application process which are described here:

-Paris-Sud and Mines' program page provided information on how to submit applications (online) -Lyon 1's program page provided the list of required documents to apply emlyon's program page provided information on important deadlines -HEC's program page provided a step-by-step description of the process

Navigation Patterns to Access the Application Portal

After examining the program page to see what information was available on the application process, this section looks at the navigation paths identified to access the application portal.

Depending on the form, there was either a direct or an indirect navigation path to the application portal (Figure 10).

Application Portal's Content

In this section, attention is given to the application portal which is the last step before the form. In the forms that were analyzed, the portal was made of two components: a gateway screen and a sign up form, both of which are looked at in more details.

Gateway Screen Content

The first component of the application portal is gateway screens [START_REF] Enders | Designing UX: forms[END_REF]. They are the last screen before accessing the application form. They are intended to inform applicants of what they need to know about the form lying ahead and provide key information to prepare filling in the form. As seen previously with the information on the application process on the program page, the gateway screens of the forms in the study contained varying levels of information as described here:

emlyon and HEC's application portal explained the purpose of the portal -Lyon 1's application portal provided a list of frequently asked questions and documentation -GEI's website did not have an application portal -Saclay's application portal provided no information.There was only a sign up form.

Sign Up Form Content

The second component of the application portal is the sign up form. From the results, it was possible to observe the following trend: business schools tended to ask similar questions and had longer form than universities. The reason for this was their sign up forms contained more questions. Figure 11 shows the headings and questions fields found in the sign up forms. All sign up forms contained questions on identity and login credentials, the two themes previously identified. A third theme found only in the sign up form of the two business schools was education.

Identified Issues

After describing the trends observed in the suprastructure, this section presents the main issues identified:

-inconsistent levels of information on the application process -unclear labelling system to access the application portal -unclear brand association -missing components of the application portal -unrelated content in the sign up form

Inconsistent Levels of Information on the Application Process

The first issue identified was the varying levels of information on the application process. Some program pages provided no information on the application process or the information was incomplete. Mines' program page for example, only provided a link but did not inform applicants they had to file their application on an external website (Figure 12). One of users' expectation was to have detailed information on the application process before filling in the application form as this user said: I would have like the platform to detail the steps by saying, '1. You will fill in this form, 2. there will be a waiting period of several weeks and 3. you will get an email informing you your application was received. Then 4. you will be invited to come in person with your supporting documents' [...]. (P6)

The information on most websites did not match users' expectations with regards to the application process. It did not allow them to understand the process and anticipate what they needed before accessing the form.

Unclear Labelling System to Access the Application Portal

The second issue identified was the labelling system intended to guide users to the application portal was not always clear. For instance, on emlyon's website, the two call-to-action buttons leading to the application portal were hard to distinguish from one another (Figure 13). The top one talked about creating a personal space (Créez votre espace personnel) and the other about applying (Candidatez) but both buttons lead to the same page. Similarly, on Lyon 1's website, the links to access the application portal were misleading. The term "Dossier de candidature" for example, seemed to imply the form had to be downloaded. The term "Adresse web d'inscription" confused application and registration. Lastly, the term "Adresse de dépôt de candidature" seemed to imply forms had to be deposited in a repository and not that the application process was an online procedure. Yet, all links led to the same application portal. This was confusing for users and they did not always find accessing the application form to be clear as this user commented when he saw the term for registration (inscription) was used for application (candidature) in Paris-Sud's website:

"Inscription" is like registration? [Yes]. But registration is like when I get the offer. But I'm applying. Well they say "candidature en ligne" anyway. (P10)

Unclear Brand Association Between Institutions and Partner Entity

The second issue was the lack of information on the relationship between an institution and its partner entity. For example, on Paris-Sud's website, the program page is entitled "Master Biologie Santé -Université Paris-Saclay" (Figure 14) which initially confused a user who said: At the bottom of the same page, the section about the application process read: "The first step to prepare for registration at one of Université Paris-Saclay's master degrees is to apply online." [free translation] (Figure 15). When users clicked on the hyperlink, they were redirected to Paris-Saclay's admission page (Figure 16). Even though there was an icon indicating the link lead to an external site, a user was surprised to land on a different website:

The link directed me to a general page whereas I had already selected 'master Biologie-Santé'. I expected to be directed to the platform where students apply, that the page would remain green you know [returns to previous page]. [...] now it sends me to an Admission page that is no longer specific to the program. (P8)

Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website

Lack of Contextual Information on the Gateway Screen

The third issue identified was the absence of contextual information on the application portal's gateway screen concerning the tasks applicants could carry out through the portal such as applying to programs of study or following the status of an application. Most users understood why they had to sign up to apply but it was not obvious as this user's reaction illustrates when he arrived on Saclay's application portal (Figure 17): This user was annoyed about not being unable to access information about the form and the application process before signing up. The program page provided no information on required documents nor the application process. Paris-Saclay, as a partner entity, provided some information about the process but as Figure 17 illustrates, the only other information the user had was that signing up was mandatory to apply. Therefore, the user could not anticipate what the process would be like and what documents he needed to prepare to apply.

Unrelated Content in the Sign Up Form

The fifth and last issue identified was some sign up forms contained questions unrelated to the task users were trying to accomplish at that stage (i.e. signing up). These questions made some the forms lengthy as a user testing emlyon's said:

[...] it means they will be able to spam me for other programs, they need to collect my data. I would have preferred to give this information later but I understand why they are doing this. (P3).

This user felt the institution was fishing for his information in any way it could, notably by making questions about his education mandatory even though he would provide this information in the application form. This created a sense of distrust towards the institution and put the user in a bad disposition even before accessing the form itself.

Identified Trends in the Forms' Macrostructure

After discussing the suprastructure, this section presents the findings of the analysis of the macrostructure. This layer refers to the organization of the content inside the form, notably structure, hierarchy and organization [START_REF] Kavanagh | Anatomie et fonctionnement du schéma PANATER en architecture d'information Web[END_REF]Roberge, 2015, Bellino, 2013). These three components are important for users to know what each part of the application form is about and know where they are inside the form.

Structural Patterns

This section presents the findings uncovered when examining the navigation inside the form. The analysis helped identify two structural patterns: linear structure and hub-and-spoke structure. Application forms with a linear structure were organized in a sequential order where the user had to fill in the mandatory fields in a page to be able to move to the next page (Figure 18). In application forms with a hub-and spoke structure, each section arrayed off a central home page and was independent from the others. This meant users were free to start the application with the section of their choice.

Linear Structure (GEI and Saclay)

Hub-and-Spoke Structure (emlyon, HEC and Lyon 1)

Content Hierarchy

After looking at the structure, this section focuses on the organization pattern of content found inside the forms. Inside each application form, content was organized in distinct groups and sub-groups, resulting in a hierarchy of content that either flat or deep. In forms with flat hierarchies, a section addressed one topic and contained no or few subheadings (Figure 19). In forms with relatively deep hierarchies, on the other hand, a section could contain several subheadings.

Flat Hierarchy (Lyon 1) Deep Hierarchy (emlyon)

Content Organization

After considering the forms' structure and content hierarchy, attention is given to the labelling system inside each form. The focus was placed on the headings and subheadings delineating groups and subgroups of content. What emerged is the forms used a variety of headings to designate identical or similar content groups.

For instance, depending on the form, the questions about education were organized in one section or dispersed in several sections (Figure 20).

Identified Issues

After describing the form structure, content hierarchy and organization, this section describes the issues that were identified. Firstly, the form's structure did not match what users expected to see in some instances. Secondly, forms with deep content hierarchies tended disliked by users. Lastly, forms often suffered from confusing section headings and subheadings.

Forms Structure Differing From Users' Mental Model

The first issue identified was the mismatch between the application form's structure and how users expected the form to be organized. From user interviews, it was possible to tell users expected an application form to follow a linear structure based on their past application experiences. Hence, they might not respond well if the form did not have a layout following this logic which was the case with emlyon's form (Figure 21) which is organized in tiles. This prompted the following reactions from users testing the form:

I don't like it. Not having something linear is weird. (P3) I wasn't expecting this. I was expecting questions. (P6)

These comments seem to indicate users did not respond well to this layout since it did not match their mental model. They expected the form to follow a linear structure with a clear beginning and end which was not the case with this form.

Overlapping Content Categories

The second issue identified was the presence of overlapping categories in some forms. This was often the case when a form contained many layers. For instance, in emlyon's form users had to manually enter information about their previous and current studies under the tile Ma candidature and then, under the tile Mes diplômes, they had to enter information about their diploma which was essentially the same. This visibly frustrated a user who commented:

They already asked me what I had studied, where and with the grade point averages. [...] I don't understand why I have to fill this in again. Either you put the part with the grade averages here or you put this part over there. [...] The two should be merged one way or another. (P3)

The second user who tested the form wondered what the distinction was between the two headings saying:

It's a duplicate and they have already asked me this information under "Candidature". [...] But I'm wondering what the difference is between the two. (P6)

The users were not always able to see the amount of information contained in each section. They became annoyed when they noticed overlaps because it meant providing the same information again and that there was no communication between the sections of the form. Figure 22 shows screenshots of the two sections. Identical field labels were identified with the same colour in both screenshots to show redundant fields.

Ma Candidature (emlyon)

Mes Diplômes (emlyon)

Confusing Headings and Subheadings

The third and last issue identified was the headings and subheadings were often unclear or ambiguous. The main reasons for this confusion were:

-the absence of headings in certain sections -headings and subheadings used as catch-all categories (Figure 23) -headings and subheadings hard to distinguish from one another -headings and subheadings unrelated to the content they were associated to After identifying trends in the macrostructure, this section presents the findings of the analysis of the forms' microstructure. The microstructure refers to question fields inside a section of the form [START_REF] Kavanagh | Anatomie et fonctionnement du schéma PANATER en architecture d'information Web[END_REF]Roberge, 2015, Bellino, 2013). It focuses on the order and layout of the fields and how the questions are worded. The findings focus on two sections: identity and education which were common to all application forms in the study and are maybe the most essential ones.

Identity Section

This section presents the findings of the analysis of the identity section and focuses on the layout of the fields in the section and how questions fields were worded.

Fields Layout and Resulting Reading Patterns

This section looks at the layout of the fields in the identity section. In all application forms, even those that had a hub-and-spoke structure, identity was usually the section that appeared first. It was also the first section users expected to find in the application form so it seemed logical to start with this section. The number of fields in this section ranged from 11 fields to 15 fields and they were arranged in various layout resulting in three reading patterns (Figure 24):

-the reversed N-shaped pattern, where fields are arranged in two columns. The applicant filled in the fields in the left column from top to bottom and repeated the same pattern with the right column.

-the I-shaped pattern, where fields were arranged in one column which the applicant filled in from top to bottom.

-the Z-shaped pattern, where fields were arranged in multiple columns. The applicant filled in the fields in the first line from left to right, then went down to the line below, started from the first field on the left and continued to the last one on the right.

Reverse N-Shape (emlyon) I-Shape (HEC and Lyon 1)

Z-Shape

(GEI)

Question Fields

This section presents the two common content groups that emerged of the analysis of common question fields in the identity section: applicants' names and biographical information (Figure 25).

Question Fields Concerning Applicant's Names

Three trends were identified in question fields about applicants' names. First, the question fields always followed the same order: last name first, then first name. Second, question fields about the first name used the same term, prénom. Third, question fields concerning the last name were divided in two subgroups:

question fields about an applicant's last name at birth (e.g. nom, patronymique, de jeune fille)

question fields about an applicant's current last name (e.g. nom, nom marital, usuel, d'usage, de famille) Before identifying the trends that were observed, it is important to understand what is meant by biographical information. The term refers to fields about personal titles, national student identifier, birth, citizenship and marital status. Two trends were identified and are described here:

-most forms required personal titles and used the same question field, Civilité -most forms asked for the national student identifier, INE (identifiant national étudiant) and referred to it by its acronym -question fields about the place of birth focused on three geographic entities: country (pays), département (an administrative division in France) and city (ville) -question fields about citizenship were generally placed at the end of the identity section and used the same word, citoyenneté. Most forms asked about dual citizenship using slight variations such as Autre nationalité or Double nationalité

In addition, the fields for the date of birth were arranged following one of the following layouts (Figure 26):

Education Section

This section presents the findings of the analysis of the education section mainly, the layout of the fields and how questions fields were worded.

Fields Layout and Resulting Reading Patterns

This section looks at the arrangement of the fields in the section about an applicant's current and past studies. This information helps an institution determine if an applicant meets entry requirements and assess their suitability for the program. The number of fields in this section ranged from 12 fields to 33 fields and the layout of the fields resulted in three different reading patterns (Figure 27):

-the I-shaped pattern, where fields were arranged in one column. The applicant filled in the fields from top to bottom.

-the Z-shaped pattern, where fields were spread in multiple columns. The applicants filled in the fields from left to right in the first line, then went down to the line below and started from the first field on the left to the last one on the right, repeating this motion for every line.

-the combo, where fields combined the I-shaped and Z-shaped patterns. The applicant started with fields aligned in a single column and switched to a multicolumn layout.

Combo

Question Fields

This section presents the trends identified when analyzing question fields in the education section which are described here:

-there were two common content groups: baccalauréat, the academic qualification French students take when they finish high school, and post-secondary education, the studies after baccalauréat -the content groups usually followed a chronological order: baccalauréat first and post-secondary studies, second

In addition, there were specific trends in each of the two content groups which are described here:

-question fields concerning baccalauréat studies generally focused on three elements: content (série), year of graduation (année d'obtention) and level of honours (mention) (Figure 28) -questions fields concerning post-secondary education focused on four elements: field of study and area of specialization, year of graduation, name of the institution attended and academic grading (Figure 29)

Identified Issues

After describing the trends observed in the sections about applicants' identity and education, the identified issues are described. First, some the layouts created confusing reading patterns. Secondly, some questions were deemed intrusive, inappropriate for the context or unadapted to users. Thirdly, users did not always understand how to use date pickers.

Inconsistent Interpretations of Fields Induced by Multi-Column Layout

The first issue concerned the different reading patterns resulting from the arrangement of fields. When there are multiple columns, users can be confused due to the many ways of interpreting how fields relate to one another. Research findings on the usability of forms [START_REF] Appleseed | Synthèse du Baromètre des métiers de la communication de l'enseignement supérieur[END_REF] revealed users generally have a different understanding of how the fields have to be filled in than what is initially intended when the form is arranged in multiple columns. Multi-columns layout also requires more coordination of the mouse, hand and eyes and most users do not use the tab key on the keyboard to move from one field to another.

Sensitive Subjects Dealt With Incorrectly in Forms

The second issue concerned the manner in which form dealt with sensitive concepts such as marital status, date of birth and citizenship. These concepts refer to highly personal subjects but were not always dealt with correctly in the forms. For example, this user felt asking applicants' their marital status was inappropriate for the context:

If I was a woman and they asked me my marital status, I would be frustrated. (P3)

The Similarly, questions about citizenship raised concerns for some users. One user, for example, said she would not disclose her citizenship if she felt she could be discriminated:

If I was from Tajikistan, maybe I wouldn't say it, I would be afraid of being discriminated against. Is it the same application for foreign students? This user felt it was important for the institution to have a way of identifying prospective students who were unfamiliar with the French education system so they could receive personalized support but sensed it could raise concerns.

Confusing Date Fields

The third issue was fields intended for the date of birth were often not intuitive, especially when they used a date picker. emlyon, HEC and Lyon 1's forms used a date picker combined with a single field which allowed users to select a date in the calendar or to type in their date of birth directly. However, none of the users realized they could type in the date directly. They thought they had to use the date picker which prompted the following reactions: Although there was a field which allowed users to type in their date of birth directly, it was not visible for users. In addition, two of the three forms had no help text with the required date format if users had wanted to type in the date.

Questions Unadapted to Different Educational Profiles

Another similar issue with the questions concerning applicants' past or current postsecondary education was they were not adapted for applicants with non-linear education trajectories. For instance, GEI's form focused on the first three years of study after obtaining the baccalauréat. A user who had completed more than three years of study commented: Another user felt HEC's form didn't allow her to specify the type of institution she had attended which could lead to her academic grading being misinterpreted:

Having 12 in preparatory classes is like getting 16 in university. [...] The two grades don't mean the same thing. However, the fact that it is a preparatory class isn't taken into consideration. (P4)

Questions in this section worked well for students who had followed what could be deemed a classical trajectory: baccalauréat and undergraduate studies. However, for users who had studied in different fields or cumulated many years of studies for example, this section did not allow them to input all the information.

Recommendations to Address Identified Form Issues

After sharing the trends found in each layer of the forms and identifying the issues uncovered through the corpus analysis and user tests, a series of recommendations addressing these issues identified are formulated in this section (Figure 30). They are categorized by layer and cross-referenced with the issues previously identified.

For the suprastructure, the following recommendations should be considered:

1. Providing a Complete Description of the Application Process (refer to 7.1.4 Inconsistent Levels of Information on the Application Process)

The program page should provide a step-by-step description of the application process so applicants understand the requirements and the procedure. The labels intended to lead applicants to the application portal should make it explicit the application is an online procedure so users know what to expect and distinguish the concepts of application and registration. The sign up form should be short and only ask the information required for applicants to access the form (e.g. first and last name and email).

Explaining Joint Partnerships

For the macrostructure, the following recommendations should be considered: A section should deal with one topic at the time. In addition, the naming convention should be unambiguous so applicants can understand the topic of a section.

For the microstructure, the following recommendations should be considered:

1. Arranging Fields in a Single-Column Layout (refer to The fields in a form should be aligned into one column so as to remove any potential ambiguity with regards to the order in which fields have to be filled.

2. Removing Questions Unrelated to the Application Process (refer to 7.

Sensitive Subjects Dealt With Incorrectly in Forms)

The form should only contain questions pertaining directly to the application process and avoid sensitive subjects unrelated to the application. In instances where such information is necessary, it is important to explain the purpose of the question and how this information will be used to ease concerns about discrimination. Program requirements, information on eligible profiles and the basis for assessing applicants' eligibility should be clearly stated (e.g. three years after the baccalauréat, the last diploma, all studies pursued by the applicant).

Using

Conclusion

The objectives of this thesis were to understand the structure of university application forms for graduate studies in France, users' past application experiences with the admission process to undergraduate and graduate programs of study and use this information to make recommendations.

Interviews revealed users' past application experiences applying for undergraduate and graduate studies often didn't meet their expectations or address their concerns. They felt information on the application process was incomplete or hard to find, especially when they did not have a centralized system such as Admission Post-Bac. In addition, there was a lack of feedback throughout the process, which raised concerns. Generally, users tended to have mixed feelings or negative views of their past application experiences.

The corpus analysis and users tests identified issues with the application forms' structural patterns. In the suprastructure, it revealed navigational paths to the application form were not always clear. The analysis of the macrostructure highlighted the content structure, hierarchy and organization inside application forms wasn't always logical nor easy to understand. Finally, the analysis of the forms' microstructure which focused on sections about identity and education pointed out how the arrangement of fields could make filling a form laborious and how certain questions were considered intrusive or unadapted to the audience.

Following these findings, the recommendations to improve user experience in each layer consisted in the following:

Suprastructure

Supra. 1: providing a complete description on the application process Supra. 2: ensuring the navigation path to the application portal is clear Supra. 3: identifying partner entities and their association with the institution Supra. 4: explaining what applicants can do through the application portal Supra. 5: ensuring the sign up form only contain the essential questions to register

Macrostructure

Macro. 1: structuring the form's layout following a sequential logic Macro. 2: dealing with one topic and using specific section headings

Microstructure

Micro. 1: arranging fields in a single column layout Micro. 2: removing questions unrelated to the application process Micro. 3: using single or separated fields for the date of birth instead of date pickers Micro. 4: ensuring questions on studies are adapted to different educational profiles

This study revealed many insights by choosing to focus on users' experiences with university application forms. That being said, it also overshadowed some issues which could be the subject of future research. For instance, the institutional perspective on forms, especially the process surrounding the creation and management of university application forms, how answers are used and the business value of every answer is a subject that could bring out important insights in handling application forms. It could also help to improve the analysis of these types of documents in French higher education.

Another foreseeable subject for future research would be how university application forms can be better integrated in the admission service chain. One of the arguments in this study was that issues with forms mainly had to do with a lack of understanding of the form as a conversation rather than a matter of technology or aesthetics. This conversation takes place within the broader process of the admission process which starts when an applicant visits an institution's website and finishes when they become are enrolled as students. Yet, as the analysis of the suprastructure has shown, the navigational paths to the form and the form itself are not always consistent with one another. Using service design to develop a more integrated approach to the different steps of the admission process would help institutions improve service delivery and user experience.

Figure 1 -

 1 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 5 -

 5 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 7 -

 7 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 12 -

 12 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 20 -

 20 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 23 -

 23 Figure 1 -Phases of the Admission Process ....................................................................... Figure 2 -SPA's Flowchart for the Applicant Experience (2017) ......................................... Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization ................... Figure 4 -Screenshot, Analysis Grid to Assess Application Forms ..................................... Figure 5 -Users' Expectations During the Application Process ........................................... Figure 6 -Users' Concerns During the Application Process ................................................Figure 7 -Users' Views of their Past Application Experiences ............................................. Figure 8 -Users' Expectations and Concerns and Improvement Recommendations .......... Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure .......... Figure 10 -Representation of the Direct and Indirect Navigation Paths .............................. Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms ..Figure 12 -Screenshots, Application Section on Mines' Website ........................................ Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites ................... Figure 14 -Screenshot, Header of Paris-Sud's Website ..................................................... Figure 15 -Screenshot, Application Section on Paris-Sud's Website .................................. Figure 16 -Screenshot, Admission Page on Paris-Saclay's Website .................................. Figure 17 -Screenshot, Paris-Saclay's Application Portal ................................................... Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure .............. Figure 19 -Representation of a Flat and Deep Hierarchy ................................................... Figure 20 -Representation of Headings and Subheadings for Certain Sections ................. Figure 21 -Screenshot, Central Home Page in emlyon's Form ........................................... Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form ..Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form ...... Figure 24 -Representation of Reading Patterns, Identity Section ....................................... Figure 25 -Representation of Common Fields, Identity Section .......................................... Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields ............................. Figure 27 -Representation of Reading Patterns, Education Section ................................... Figure 28 -Representation of Common Question Fields, Baccalauréat Studies.................. Figure 29 -Representation of Common Question Fields, Post-secondary Education .......... Figure 30 -Issues Identified With the Corpus Analysis and User Tests and Recommendations ............................................................................................

Figure 1 -

 1 Figure 1 -Phases of the Admission Process

Figure 2 -

 2 Figure 2 -SPA's Flowchart for the Applicant Experience (2017)

Figure 3 -

 3 Figure 3 -Screenshot, Grid Created to Map Forms' Content and Organization

Figure 4 -

 4 Figure 4 -Screenshot, Analysis Grid to Assess Application Forms

Figure 6 -

 6 Figure 6 -Users' Concerns During the Application Process

  It's practical because there's one form, we all have the same [...]. It's a universal door I would say, so it's reassuring. I'm not wondering, 'Did I miss something? Maybe that's not how I should have applied'. (P6)

I

  don't think I had asked [for help]. I think I didn't want to show I was a little lost. (P9)

Figure 7 -

 7 Figure 7 -Users' Views of their Past Application Experiences (Positive Views in Green, Negative Views in Red and Neutral Views in Light Yellow)

  It's faster because it's online but I still preferred the first time [with APB]. I found it more reassuring to have someone do it for you because then it cannot be wrong. (P2)

Figure 8 -

 8 Figure 8 -Users' Expectations and Concerns and Improvement Recommendations

Figure 9 -

 9 Figure 9 -Representation of the Suprastructure, Macrostructure and Microstructure 7.1 Identified Trends in the Forms' Suprastructure

  Figure 10 -Representation of the Direct and Indirect Navigation Paths

Figure 11 -

 11 Figure 11 -Representation of Headings and Common Question Fields in Sign Up Forms

Figure 12 -

 12 Figure 12 -Screenshots, Application Section on Mines' Website

Figure 13 -

 13 Figure 13 -Screenshots, Labelling System on emlyon and Lyon 1's Websites

  No, this is not the correct university, is it? [highlights the term "Paris-Saclay" in the page title]. Ah no, it's got the logo here [points to upper left corner of the screen]. So Paris-Sud is like Paris-Saclay I guess. Or maybe it's like one of them. (P10)

Figure 14 -

 14 Figure 14 -Screenshot, Header of Paris-Sud's Website (Mentions of Paris-Sud in red and those of Saclay in blue)

  [...] I wasn't expecting I would have to create an account for a place where I'm not even sure I will be admitted. Because that means if I'm not admitted it will be easy for them to sell my data [...] (P8).

Figure 17 -

 17 Figure 17 -Screenshot, Paris-Saclay's Application Portal

Figure 18 -

 18 Figure 18 -Representation of the Linear Structure and Hub-and-Spoke Structure

Figure 19 -

 19 Figure 19 -Representation of a Flat and Deep Hierarchy

Figure 20 -

 20 Figure 20 -Representation of Headings and Subheadings for Certain Sections

Figure 21 -

 21 Figure 21 -Screenshot, Central Home Page in emlyon's Form

Figure 22 -

 22 Figure 22 -Screenshots, Sections Ma Candidature and Mes Diplômes in emlyon's Form

Figure 23 -

 23 Figure 23 -Screenshot, Catch-All Category (Données personnelles) in emlyon's Form

Figure 24 -

 24 Figure 24 -Representation of Reading Patterns, Identity Section

Figure 25 -

 25 Figure 25 -Representation of Common Fields, Identity Section Question Fields Concerning Biographical Information

Figure 26 -

 26 Figure 26 -Screenshots, Date Picker, Single Field and Separate Fields

Figure 27 -

 27 Figure 27 -Representation of Reading Patterns, Education Section

Figure 28 -

 28 Figure 28 -Representation of Common Question Fields, Baccalauréat Studies

2.

  Ensuring the Navigation Path to the Application Portal is Clear (refer to 7.1.4 Unclear Labelling System to Access the Application Portal)

  to move on to the next step even if

Table 1 -

 1 Institutions Included in the Corpus Analysis

	Institution Name	Institution Type	City
	emlyon	Business School	Écully
	HEC Paris	Business School	Jouy-en-Josas
	Mines Saint-Étienne	Engineering School	Saint-Étienne
	Université Claude Bernard	University	Villeurbanne
	Lyon 1
	Université Paris-Sud	University	Orsay

Table 2 -

 2 Institutions and Program Names of Examined Application Forms

  user didn't see what the marital status had to do with the application and even though it was optional, he didn't feel this question was appropriate in this context. Similarly, another user shared how uncomfortable giving his date of birth made him feel:This question is a little sensitive because for me it's discriminatory. [...] In France, people do their master when they're really young, most are 22 or 21 years old so it bothers me to write 1988. [...] Again, it's like job interviews, when someone asks your age. You know there is something wrong. [...] I would have the same feeling: Are they asking my age because they think I'm too old? (P9)

  [Yes] You cannot really distinguish then [between French students and foreign students]. You can be a foreign student, have French citizenship but have lived abroad your entire life and be coming to France [for the

first time]. That is not very clear then. It's not a problem that they are foreign students, but I think it's good that we know they haven't studied in France before..(P6)

  I prefer to type it [my date of birth] than search in a calendar honestly. It's nice when you're booking a train but not when it's your date of birth which you already know. (P3) I have to go down like this?! [User then proceeded to click on the arrows of the calendar to reach her year of birth, month of birth and select her day of birth] (P2)

  When the Application Process is Carried Out Through an External Website (refer to7.1.4 Unclear Brand Association Between Institutions and Partner Entities)When the application process is carried out through a partner entity's website, institutions should ensure they identify this entity and explain its role in the process.

	4. Explaining the Purpose of the Application Portal
	(refer to 7.1.4 Lack of Contextual Information on the Gateway Screen)
	The application portal should explain what and who the form is for, required
	documents, estimated completion time and legal considerations.
	5. Ensuring the Sign Up Form Only Contain Necessary Fields
	(refer to 7.1.4 Unrelated Content in the Sign Up Form)

  Single or Separated Fields for the Date of Birth (refer to7.3.3 Confusing Date Fields) The date of birth is something applicants usually know by heart. It is therefore more efficient to use text fields in which applicants can type in the information than a date picker which requires applicants to search for the correct date. For clarity, fields should also be accompanied by help text explaining the date format required.

	4. Ensuring Questions About Studies Are Adapted to Different Educational Profiles
	(refer to 7.3.3 Questions Unadapted to Different Education Profiles)

Acknowledgments

Acknowledgments Abstract

Identified Issues

Recommendations