Network operators and service providers from the very advent of telecommunications wanted to know what is the level of service quality which is provided to the end users. This is because that knowledge can be extremely useful when trying to manage network topology, optimize its capacity and operating costs, introduce new services or plan investments and expansion of a network.
In general, we can look at the service quality from two very different, but interconnected, perspectives. On the one side, there is a network whose performances are usually determined with network related parameters. Nowadays, we are talking about the packet switched multiservice networks, such as the Internet, that can be evaluated with performance indicators such as network delay (latency), jitter, packet loss rate, bandwidth, throughput and error rate. Note that it is common to call these parameters objective parameters because they are measurable and quantitative. If you look at the pre-QoE era scientific papers, you will find that most authors denoted the network performances (measured by the objective parameters) as Quality of Service (QoS).
On the other side of the service quality food chain, there stands a user. Ignorant of network properties and its performances; only asking a satisfactory service quality for a fair price. For most users, objective parameters are usually unknown terms defined by a scientist and engineers in some laboratory far, far away. Their only concern is how a service performs on the application level and they evaluate the service quality through the qualitative subjective parameters such as level of satisfaction, enjoyment, entertainment, immersion, annoyance, interface quality and design, service usability and others.
So now we know that the service quality has some objective and subjective properties. Where is the connection between the two? Well, a user will hardly be satisfied if a network performances (QoS) are poor. For instance, if the re-buffering of a video is frequent during the streaming session a user will most certainly be annoyed and unsatisfied. Hence, network operators have to work on achieving the QoS targets (e.g. packet loss rate must not exceed 0.1%) in order to keep a user satisfied. Yet, several authors showed that achieving the QoS targets does not necessarily ensure satisfied users. Something was still missing.
Even through a network and all services are developed for the end users, in the pre-QoE era, a user was only a bystander. The quality was usually evaluated through QoS prism and even if a user was included in the process of service quality evaluation, the evaluation itself was almost always done in the artificial, laboratory environments dictated by different recommendations of the international organizations. In such highly controlled environments designed specifically to rule out all external influential factors (such as surrounding noise during voice quality testing), the subjects were more or less well informed Ginny pigs. They were asked to evaluate a service one or multiple times under same or different test conditions, sometimes instructed to focus on a specific aspect of a service quality. Moreover, they found themselves in the unfamiliar environment (laboratory), interacting with the unknown people and monitored by some eager laboratory assistant. It is understandable to find that some researchers concluded that the results of this kind of tests cannot give a complete and accurate insight into the quality of a service from a user standpoint (see paper 1 and paper 2) .
Therefore, it was time for one hard turn toward the user perspective of service quality and away from mere QoS evaluation. A new concept was developed – Quality of Experience (QoE). First of all, let us browse through some QoE definitions:
Lopez et al. defined QoE as:
“an extension of the traditional QoS in the sense that QoE provides information about the delivered service from an end-user point of view.”
Soldani et al. stated that:
“QoE is how a user perceives the usability of a service when in use – how satisfied he/she is with a service in terms of, e.g., usability, accessibility, retainability and integrity.”
International Telecommunication Union (ITU) defines QoE as:
“the overall acceptability of an application or service, as perceived subjectively by the end-user.”
The European Telecommunications Standards Institute defined QoE as:
“user perceived experience of what is being presented by a communication service or application user interface.”
According to the Digital Subscriber Line (DSL) forum QoE is:
“the overall performance of a system from the point of view of the users. QoE is a measure of end-to-end performance at the services level from the user perspective and an indication of how well the system meets the user’s needs.”
In 2010 Möller provided a more hedonistic definition of QoE:
“Degree of delight of the user of a service. In the context of communication services, it is influenced by content, network, device, application, user expectations and goals, and context of use.”
Group of experts of the European Network on Quality of Experience in Multimedia Systems and Services accepted and expanded the Möller’s definition by concluding that:
“Quality of Experience is the degree of delight or annoyance of the user of an application or service. It results from the fulfilment of his or her expectations with respect to the utility and/or enjoyment of the application or service in the light of the user’s personality and current state.”
From these definitions, two things are clear: (1) there isn’t one, universal definition of QoE and (2) QoE is far more than keeping track of QoS parameters.
QoE is a holistic way of looking at the service quality. QoS is still an important and necessary factor, but it is certainly not the only one. As we see from these definitions the focus is on end-user point of view.
In the context of QoE4VR project, this means that the VR applications must be evaluated not only through objective parameters but also through a wide set of subjective measures.
It is crucial to focus on user perspective while changing network conditions degrade the performance of a telecommunication service. Instead of counting errors during e.g. video streaming in VR environment, our test subjects have to evaluate their level of satisfaction, enjoyment, exitment etc.
Apart from the importance of the end user opinions, we cannot neglect that the test environment has to be more appealing and inviting compared to the rigid laboratory environments. Only then the users can experience that ‘home’ environment while participating in the experiment. Therefore, the QoE4VR project will create a lifelike settings in the laboratory where the test subjects’ perception will be examined.
For those who want to find out more: