The Software for OMAP5 Platform Remote Control Tester

(1)

The Software for OMAP5 Platform Remote Control Tester

Peijun Zhou

Author: Peijun Zhou

School: HZ University of Applied Sciences

Program: Bachelor of Engineering in Mechatronics Supervisor teacher: D.O. Heineke

Company: Innolux - TPO Displays Europe B.V.

In-company mentor: Maurice Essers Date of publication: 07-07-2014

Place of publication: Heerlen, The Netherlands

(2)

The Software for OMAP5 Platform Remote Control Tester version: 5.2

The Software for OMAP5 Platform Remote Control Tester

Final report

Peijun Zhou

Author: Peijun Zhou

School: HZ University of Applied Sciences

Program: Bachelor of Engineering in Mechatronics Supervisor teacher: D.O. Heineke

Company: Innolux - TPO Displays Europe B.V.

In-company mentor: Maurice Essers Date of publication: 07-07-2014

Place of publication: Heerlen, The Netherlands

(3)

Summary

The target of this project is to develop the software for the existing hardware to check the quality of the LCD display. The hardware contains a computer, platform and display. This platform was designed by TPO Displays Europe B.V., which provides an interface for showing the image on the display.

The software is divided into two parts: the PC application and the embedded software.

And this report introduces the process of designing these two parts of the software followed the methodology called “Engineering Design Process”. The PC application should run on the computer for sending pictures and movies to the platform. And it also need give some instructions to control the platform to display the image. On the other hand, the embedded software is running on the platform, which is responsible for receiving the files and executing the operations according to the instructions.

Before programming, the designer should select a suitable operating system for the platform to run the embedded software. The communication between the computer and the platform need be chosen as well. Last but not least, the PC application and the embedded software should establish a connection method called “socket” to communicate with each other. The PC application is developed in C# and the embedded software is programmed in C language.

After 4 months’ work, the communication between the PC application and the embedded software established well, and the software was designed successfully. All the bugs are corrected and improved in the test phase. However, the configuration of the interface, which makes the image shown on the display, could not be finished in this project.

(4)

The Software for OMAP5 Platform Remote Control Tester Version:5.2

Foreword

This report is a part of the result for getting the Bachelor degree in Electrical Engineering in Hz University of Applied Sciences.

First, I would like to acknowledge my company supervisor Maurice Essers, who gave the great support in developing and teaching knowledge. He always guided me using different way to solve problems, which gave me deep comprehension to my

assignment.

Then I need to acknowledge my serious and responsible school’s tutor D.O. Heineke, who was enthusiastic to solve my problem and showed me how to put my internship work into the report.

I also feel greatly indebted for other colleagues that they all encouraged me to conquer obstacles and gave me kindness help.

Also thanks for TPO Displays Europe B.V. giving me a good platform and atmosphere to enjoy my internship time.

The report is divided into several chapters below:

Chapter 1: Introduction

The reader can get an introduction about the company and the assignment. This chapter raises one main question, several sub questions and some requirements given by the company about the project.

Chapter 2: Theoretical framework

In chapter “Theoretical framework”, all the necessary and relevant knowledge and concept about this project are shown.

Chapter 3: Methodology

This chapter contains the structure of methodology (EDP) that will be applied in the project.

Chapter 4: Results

In chapter 4, it shows the prototype result of the project with the methodology introduced in Chapter 3.

Chapter 5: Test results

Test the prototype to find whether there exist some bugs or mistakes in order to upgrade. This chapter shows the test results.

Chapter 6: Conclusion

In this chapter, it gives a conclusion of the whole project.

(5)

Chapter 7: Recommendation

This chapter lists some recommendations, which can be improved in the future.

Appendix:

The appendix includes the method of installing the operating system, the code of the PC software and the embedded software, some formulas about calculating the timing parameters and a manual for setting up the development board.

(6)

Abbreviation

! CSI: Camera Serial Interface

! DSI: Display Serial Interface

! DDR: Double Data Rate

! EDP: Engineering Design Process

! GUI: Graphical User Interface

! HDMI: High-Definition Multimedia Interface

! LCD: Liquid Crystal Display

! MIPI: Mobile Industry Processor Interface

! OLED: Organic Light-emitting Diode

! PC: Personal Computer

! PLL：Phase Locked Loop

! RAD: Rapid Application Development

! RGB: Red Green Blue

! SSH: Secure Shell

! TFT: Thin-Film-Transistor

! TPO: TPO Displays Europe B.V.

! TCP: Transmission Control Protocol

(7)

1. Introduction

1.1 Company

CMI is a merger between Chimei, Innolux and TPO established in March 2010. In 2013 the name changed from CMI to Innolux.

Toppoly Optoelectronics Corp and Philips Mobile Display Systems – was established in June 2006 to create a new powerhouse focused on small and medium-size mobile displays (called TPO)^[1].

Innolux offers a highly competitive product and technology portfolio of distinctive, value added displays for the telecommunications, consumer electronics, automotive, information display and avionics markets. Headquartered in Chunan, Taiwan, Innolux is one of the largest TFT manufacturers in the world with ~110,000 employees. The Innolux international networks comprises over 10 offices in 9 countries and territories, operating in the Asia-Pacific region, Europe and the Americas.

The European headquarter in Heerlen is a key advanced product and technology development center for Telecommunication and Automotive application products, after the merge in 2010 it kept the name “TPO Displays Europe B.V.”. It also defines the global approach from Automotive and provides a strong and established support infrastructure for European sales offices^[1].

Innolux produces Liquid Crystal Displays (LCDs), Organic Light-emitting Diode (OLED) Displays and Touch Screens in the range from LCD-TV to displays for the mobile market (Telecommunication and Automotive). In Heerlen they mainly develop displays for Nokia, Research in Motion, Acer, Audi, BMW, Porsche and Volkswagen.

And now, Innolux is No.3 in the Small/Medium display market, No.2 in the mobile phone display market and No.1 in the automotive market^[1].

(10)

1.2 Determined the quality of the LCD display

In TPO Displays Europe B.V., the engineer uses test pictures and movies to determine the quality of the LCD display. The engineer can check whether there are some pixel defects, flickers and spots on the display through showing the test pictures.

When the display is in movie mode, the motion artifacts that cannot be detected by static test pictures will occur. So the engineer selects a movie to check these artifacts.

1.3 Assignment introduction

The company wants to use an existed platform for determining the quality of the LCD display. Figure 2 shows the overview of the hardware.

Figure 2 Overview of the hardware

1.3.1 OMAP5432-EVM Development Board

OMAP5432-EVM development board is the latest development board designed by Texas Instruments. The key of the OMAP5432-EVM is OMAP5432 ES2.0 processor, which is based on a Cortex-A15 core. With this processor, the development board supports some high level operating systems like Linux and Android^[2].

The operating system will be installed into a SD memory card. Then the development board will boot the operating system from the SD card when it is powered on.

(11)

Figure 3 Bottom view of Omap5432-EVM

1.3.2 OMAP5forLux

The LCD displays for mobile phones use the Display Serial Interface (DSI) as a communication interface. Though the development board has the DSI interface, it has not a connector of the DSI. So the company designed an expansion board with a DSI connector called “OMAP5forLux”. Then the image can be shown on the LCD display modules via the DSI^[3]. This expansion board is connected to the development board through the “Main Expansion connector” on the board (shown in Figure 3).

(12)

1.3.3 Assignment

According to the hardware, the company needs to set up a communication between the computer and the platform. And then develop two applications: the PC application and the embedded software. The PC application should run on a computer with Windows operating system for sending test pictures, movies and commands. The commands include displaying the pictures and playing a movie. The embedded software will be run on OMAP5432-EVM development board to receive files and commands. Then according to the commands, show the pictures or play a movie on the display.

1.4 Specific requirements

This chapter lists the requirements about the project from the company:

" Select and install a suitable operating system for the development board.

For any software, it should only work under a suitable operating system. This should be done before programming.

" The PC application is required to control more than one platform at the same time.

There are millions of the LCD displays that should be tested in the factory. If the operator tests the LCD one by one, the efficiency of checking is quite low. So control multiple platforms at the same time can speed up checking the display.

" The PC application needs a Graphical User Interface (GUI) for operators to use.

The GUI should include four parts at least: connection part, sending files part (pictures and movies), showing pictures part and playing the movie part.

" Configure the DSI interface.

Though the company has designed the hardware of the DSI interface connector, the software part of the DSI needs to be modified in this project.

" All the operations on the platform should be done automatically after the platform powers on.

The operator in the factory only needs to power on the platform. Then the work like starting the embedded software, connecting to the computer should be done automatically. The operator will not do any extra operations on the platform.

(13)

1.5 Questions of the assignment

The main question of the assignment:

" How to develop the embedded software and the PC application for the existing hardware to determine the picture quality of an LCD display?

The sub question of the project:

" What is the best operating system for the development board?

This question will be answered in chapter 4.1: Operating system selection.

" How to set up a communication between the computer and the platform?

" How to control more than one platform at the same time?

This question will be answered in chapter 4.2: Communication method.

" What are the functions of the GUI?

" How does the PC application send the files and commands?

These questions will be answered in chapter 4.3: The PC application design.

" How does the embedded software receive the files and display pictures and movies?

This question will be answered in chapter 4.4: The embedded software design.

" What is needed to configure the DSI interface?

This question will be answered in chapter 4.6: Configuration of the DSI

(14)

2. Theoretical framework

It is necessary to know some new concepts, professional knowledge and technical methods that are related to the assignment. In this chapter the knowledge, concepts and methods will be introduced based on the assignment.

2.1 Platform

This chapter mainly explains the operating systems that can be installed on the platform. The interface that is used on the development board is introduced as well.

And the details for selecting a suitable operating system can be found in Chapter 4.

2.1.1 Ubuntu

In the assignment, Ubuntu was selected as the operating system, which was install on the development board.

Ubuntu is a distribution of the Linux operating system with Unity as its default desktop environment.^[4]Ubuntu aims to provide a latest, fairly stable operating system, which contains some free software such as picture viewer and media player. Ubuntu is a completely open source operating system, so it has a powerful community. Users can get any help from the community.

2.1.2 Android

Android is another option for installing on the development board.

Android is a Linux-based open source and freely operating system, mainly used in mobile devices, such as smart phones and tablet computers. Google and the Open Handset Alliance developed Android together. The Android operating system was originally developed for the phone by Andy Rubin. It can also be installed on some development boards^[5].

(15)

2.1.3 Universal Serial Bus (USB) interface

The development board does not have a Bluetooth interface nor a WLAN interface.

So in the assignment, no matter what kind of communication (USB, Wi-Fi or Bluetooth) was chosen, they all need use the USB interface because the platform needs an USB adapter to connect with the Wi-Fi or Bluetooth.

The two-wire USBB2 HSIC interface is connected to an SMSC USB3503A-1-GL-TR physical layer on the development board. It converts the two-wire HSIC interface to three downstream DP/DM host port. Two of the ports are connected to the dual USB Type-A connector at the position of J6. Figure 4 shows a block diagram of the USBB2 interface.^[6]

Figure 4 USBB2 Interface Block Diagram

2.2 Communication and data transmission

This chapter gives an introduction of three communication methods for the hardware and two connections between the software that can be selected in this assignment.

They are: USB, Wi-Fi and Bluetooth for the hardware. Socket and Secure Shell (SSH) could be chosen in software. The reasons for selecting are written in Chapter 4.

2.2.1 Wi-Fi communication

Wi-Fi communication was set up between the computer and the platform in this assignment.

Wi-Fi is a kind of technology that makes electrical devices like computers and mobiles do some data exchanges or connect to the Internet without any cables. Wi-Fi can also be regarded as a wireless network communication technology brands. Hold by the Wi-Fi Alliance ^[7].

Wi-Fi is a wireless local area network that enables portable computing devices to

(16)

○¹ TCP/IP: The Transmission Control Protocol (TCP) and the Internet Protocol (IP) are two basic computer networking protocols. They provides a connection about how data should be formatted, addressed, transmitted, routed and received at the destination.

8

2.2.2 USB communication

BesidesWi-Fi communication, USB communication is another option that can be established between the computer and the platform.

USB defines connectors, cables and communication protocol to use a bus as

connection, communication even power supply between the computer and electronic devices. USB communication puts data into several packets and transmits them to other device. Initially, all packets are sent to devices from the host via the root hub.

Some of the packets direct a device to send some data as a reply.^[8]

2.2.3 Bluetooth communication

The third communication method option between the computer and the platform is Bluetooth.

Bluetooth can be used as a way of wireless exchanging data in short distances. It helps the user to get rid of traditional connect methods with cables. Bluetooth implement the function of data transmission through a special radio frequency. It is very secure and eight types of electronic equipment (such as a mobile, computer headphone) can be connected at the same time.^[9]

2.2.4 Socket

USB, Bluetooth and Wi-Fi are three optional communication methods between the hardware: the PC and the platform. The socket is a method to make the PC application and the embedded software communicate with each other in this project.

The establishment of the socket is reflected in the development of the PC application and the embedded software.

Socket is used to make two parts of the software that is based on TCP/IP^○¹ protocol communicate with each other. It was appeared in the UNIX system for the first time.

But with the strengthening of the communication technology, now socket can be used between Linux to Linux, Windows to Windows even Linux to Windows. The socket consists of an IP address and a port. Generally, socket is divided into two parts:

server and client. The connection will be established between the server and the client as long as the client gets the IP address and the port of the server.^[10] The transmission between the server and the client is mutual, so there is not a clear definition of selecting a server or a client. It depends on the requirements of the software designing.

(17)

2.2.5 SSH

SSH is another option for providing a communication between the software in the assignment.

SSH was developed by the IETF Network Working Group. It is based on the application layer and the transport layer on the basis of security protocols. SSH is more reliable, designed to provide security protocol for remote login session. It protocol can effectively prevent the users to remote management of information disclosure issue. It is also a program that was originally on the UNIX system, and compensate for network vulnerabilities ^[11]. Almost all UNIX system – including HP-UX, Linux, AIX, Solaris, Digital UNIX, Irix, and other operating system can run SSH.

2.3 PC application on computer

This part introduces the programming tool and the programming language for the PC application, which is running on the computer in this assignment.

2.3.1 Microsoft visual C Sharp

Microsoft Visual C sharp (also written as C#) is a compiler tool for programmers creating applications by C# programming language for Windows operating system.^[12] In Figure 5, it is the logo of Microsoft Visual C#. In this assignment, Microsoft Visual C# is used to develop PC software with a GUI in C# for the computer.

As a part of the Microsoft visual studio developed by Microsoft. Microsoft Visual C#

enhances its interoperability with other languages such as C++. The designer can build scalable application and take advantage of multi-core processors easily and quickly with the Task Parallel Library (TPL) in the Microsoft Visual C#.

2.3.2 C# language

C# is the programming language for developing the PC application with the GUI in this assignment.

C# is a multi-paradigm programming language that encompasses strong typing, imperative, declarative, functional, procedural, generic, object-oriented (class-based), and component-oriented programming disciplines.^[13]

C# was designed as a new programming language by Microsoft Corporation based on the C and C++ languages. The programmer will be familiar with the C# as long as he has some experiences in C and C++ to do some programming. C# cooperates with the high productivity of Rapid Application Development (RAD) languages and the new ability of C++.^[14]

Figure 5

(18)

11

2.4 Embedded software for the platform

The compiler and the programming language for developing the embedded software are introduced in this chapter. The embedded software will be run under a suitable operating system in the platform.

2.4.1 GNU Compiler Collection (GCC)

GCC is a compiler system produced by GNU project for various programming languages like C++, C and JAVA.^[15] In this assignment, GCC is used as a compiler for the development of the embedded software running on the platform.

Figure 6 is the logo of GCC designed by GNU project in 1987. GCC has been ported to a wide variety of processor architectures, and is used as a tool in programming non-profit software and paid software widely. GCC is also available for most embedded platforms, including Symbian. Now most other modern Unix computer operating systems like Linux regards GCC as a standard compiler.^[16]

2.4.2 C language

C language is a programming language to develop the embedded software for the platform in this project.

C language is used as a general-purpose, high-level language that was originally developed by Dennis M. Ritchie to develop the UNIX operating system at Bell Labs. C language was implemented on the DEC PDP-11 computer in 1972. In 1978, Brian Kernighan and Dennis Ritchie published the first available description of C, which is known as the K&R standard.

The C has now become a widely used professional language for various reasons.^[17]

" C language is a programming language that is to learn.

" C language can develop efficient programs.

" C language can solve low-level activities.

" C language is suitable for different types computer platforms

Figure 6

(19)

2.5 Device under test

The LCD display is the device that needs to be tested by the software and the DSI is an interface that connects with the LCD display. Through the DSI, the image can be shown on the display.

2.5.1 LCD Display

LCD (liquid-crystal display) displays are placed in the structure in which two parallel liquid crystal cell glass substrates provided TFT (Thin Film Transistor) on the glass substrate. A color filter disposed on the upper glass substrate. Change the voltage signal on the TFT liquid crystal molecules can control the rotational direction ^[18]. Then the polarization of the emission of each pixel is controlled.

Figure 7 The internal structure of LCD’s

There is a kind of material called LC materials, which contains a liquid crystalline phase. This material is located between solid phase and liquid phase, which is regarded as an additional phase in some materials. The long cigar-shaped organic molecules in this material and the elastic coupling in molecules are so powerful ^[19]. There are two polarizing filters on the both sides of the LC layer. One of the filters polarizes the light in a vertical direction, and the other one is on horizontal direction. In fact, the purpose of the LC layer is not to produce the light. The work principle of the layers is change the direction of the long cigar-shaped organic molecules of the LC material, and then the light outside will be transmitted or reflected. As a result, the LC layers require a fluorescent or LED backlight behind it ^[20].

(20)

13

2.5.2 DSI interface

MIPI DSI (Display Serial Interface) is one of the most important interfaces for the platform. The interface will be enabled when the developer wants to drive telecom (smart phone) displays. This interface was developed by the Mobile Industry Processor Interface (MIPI) Alliance. DSI provides a communication protocol that is used between the target (the display) and the host (in this case the OMAP processor) and a serial bus ^[21].

The DSI on the physical layer called DPHY. It is also used by the camera interface developed by the MIPI Alliance, which is called Camera Serial Interface (CSI). A DPHY uses one clock lane and one to four data lanes. The advantages of DPHY are:

a) It facilitates integration and interoperability, because it is a standardized serial bus.

b) A four data lane configuration can speed up to 4 Gbps. The maximum speed of one lane is 500 Mbps.

c) The power is consumed very low.

d) High scalability: a DPHY can use one to four data lanes determined by the user.

2.5.3 Linux Kernel

The Linux kernel is a part of the Linux distributions.In the Linux kernel, the user can find all the basic and necessary programs and modules^[22]. For example: network driver, sound driver, Wi-Fi driver and the image output driver. The kernel is used to configure the DSI interface in the assignment.

The Linux kernel is written in C language and it was created in contributors worldwide in 1991. All the source of the kernel is open to the public, so a Linux user can modify the Linux kernel to create a Linux of his own.

(21)

3. Methodology

While knowing the knowledge related to the project, selecting a correct and proper methodology would provide the designer a higher efficiency and a perfect performance in a project. So the methodology used in the internship will be introduced in this chapter.

3.1 Engineering Design Process

This chapter gives a clear view of the methodology and introduces it step by step. In TPO, the colleagues use a method called Engineering Design Process (EDP) to finish a project. This internship assignment is also a part of the project in TPO. So the methodology should be same with the company.

3.1.1 Introduction

EDM can be regarded as a formulation to help a designer to realize his target. It includes several steps, and the engineer should follow the methodology step by step.

If it is necessary, some phases may repeat for more than once. Here is an overview of the methodology in Figure 8.

Figure 8 The steps of the Engineering Design Process

(22)

15

3.1.2 Define the problem

The Engineering Design Process begins at defining the problem. At the time an engineer receives a project, he should make a clear view about what this project is about. Then the main question will be got and several related sub questions for solving the main question are listed. This phase has finished in Chapter 1.

3.1.3 Do background research

Before starting a project, the designer needs to know some necessary knowledge and concepts about the assignment. The research should relay to the main question and sub questions. The platform, the communication, the PC application, the embedded software and other related devices build up a complete framework in this phase. All the knowledge and concepts related to these parts can be found in Chapter 2.

3.1.4 Specify requirements

There might be lots of requirements that come from the client, the company or other department in a project. These requirements are what the designer needs to realize.

All the requirements are listed and given an explanation in this project in Chapter 1.

3.1.5 Choose a best solution

Generally, different choices can give a designer more idea in solving the problem. The designer needs to make a comparison of all the possible selections like doing some test or combining with the real condition to get an evaluation for each option. Then the best choice can be selected. The options were given in selecting a suitable operating system and communication (sub question 1 and sub question 2) in this assignment.

The final solution will be determined through the analysis, comparison and combining with the real situation. This part can be found in chapter 4.

3.1.6 Build the prototype, test and redesign

As long as the engineer gets the solution, next phase is to make the solution come true—develop the solution. This phase requires improving the solution as well and keeps improving until the whole process of the assignment is finished.

For a complete assignment, it is always divided several functions or parts. When the designer finishes building all the prototypes, they are used to check whether it meets all the specific requirements, or do some tests.

How to check the bugs in the software and make sure the product can work correctly and steadily? The answer is testing. A test can help the designer to find which part has a mistake and find the problem, then redesign to fix it.

The designer will build some prototypes for the communication between the PC application and the embedded software, the PC application (include GUI), the embedded software and the configuration of the DSI (sub question from 2~7). The PC application will be developed in C# and C language is used to program the embedded software. The DSI is configured through calculating the timing and the frequency. The

(23)

designer can improve the result or fix the bug in the test phase. All the prototypes are built in Chapter 4 and the test phase can be found in Chapter 5.

3.1.7 Communicate the result

This is the last step of the method, which also means the task is close to complete.

The way to communicate the result should be a final report, a demo, even a presentation. For a huge assignment, it is also necessary to write a thoroughly document as a manual for the user to check.

(24)

4. Results

This chapter shows the process of designing and the results of this project. It includes five parts: operating system selection, communication selection and establishment, the PC application design, the embedded software design and the configuration of the DSI. All the sub questions will be answered in this chapter.

4.1 Operating system selection

An operating system is a basic condition for running the embedded software on the platform. As the methodology, the alternative choices of the operating system for OMAP5432-EVM development board are Ubuntu (one of the distributions of the Linux) and Android.

The way of selecting the operating system is to install Ubuntu and Android on the platform respectively and give a comparison. The result is shown in table 1.

Android Vs Ubuntu

Android Ubuntu

Desktop environment Yes Yes

Booting time 32s 84s

Wi-Fi Yes Yes

Bluetooth Yes Yes

USB mouse Yes Yes

USB keyboard Yes Yes

Stability System crash in 15 mins Stable

Table 1 Comparison between Android and Ubuntu

According to the table 1, we can find that both Android and Ubuntu have Wi-Fi module and Bluetooth module. The designer used the USB mouse to control the cursor and did some typing via a keyboard under the desktop environment in both two types of the operating systems. The development board booted Android in 32 seconds.

However, it took more than 1 minute (84s) to boot the Ubuntu. So Android is better than Ubuntu in the aspect of booting time.

To verify the stability of the system, the platforms were powered on to run Android and Ubuntu separately for an hour. It has been found that the platform running Android crashed after 15 minutes. When rebooted the platform, the system crashed again in 15 minutes. Turning to another operating system, Ubuntu ran stably on the platform after an hour without any crashes. Then the time is extended to 2 hours for a deeper test. The system still kept running on the platform and maintained a good status when using Ubuntu. As a result, Ubuntu is more stable than Android.

Conclusion: The stability of an operating system is the key consideration that should be considered. A test between the Android and Ubuntu shows Ubuntu is more stable than Android. So Ubuntu was selected as the operating system to install on the platform. The method of installing Ubuntu can be found in Appendix I.

(25)

4.2 Communication method

This chapter is divided into three parts. First part is how to select a suitable communication method. Second part introduces why choosing socket for the software to communicate and how to establish a socket. The third part is how to make the computer control more than one platform at the same time is the last part of this chapter.

4.2.1 Communication method for the hardware

A connection must be built if the PC application and the embedded software want to communicate with each other. The communication method should be easy to establish, transmit in a high speed, and meet the requirements of the company. The three optional communication methods: USB communication, Bluetooth communication and Wi-Fi communication have been introduced in Chapter 2.

According to the research and the evaluation, Wi-Fi communication is the final solution. The reasons why selecting Wi-Fi communication between the PC and the platform are listed as followed:

a) The platforms are fixed on a special round turning test desk in the factory. When the operator starts testing, the test desk will rotate in a certain direction. This will cause the USB cables intertwined. So combining with the really testing environment, only a wireless connection method can be used in this assignment.

b) Both Wi-Fi and Bluetooth can connect two or more devices without using any wires. But one megabyte per second is the maximum data transfer speed of the Bluetooth. The Bluetooth cannot be an ideal choice for data transmission because of the low transfer rate.

c) Wi-Fi communication method is the most popular connection method between two devices. The transfer speed is the fastest among the three communication methods as well. The range of Wi-Fi is also very large, a room, a floor even the whole building. So the computer can control the platform no matter where the computer is as long as the computer and the platform connect to the same Wi-Fi.

According to the above three reasons, Wi-Fi communication will be used in this project.

(26)

4.2.2 Communication for the software

The connection for the hardware is settled, and next step is to determine a proper software communication between the PC application and the embedded software.

There are two software communication options for selecting: SSH and socket.

Both SSH and socket are two popular communication mechanisms (The definitions of SSH and the socket have been given in Chapter 2). But the SSH prefers to serve for the UNIX system like Linux. So for Windows operating system, it needs plenty of work to create a complicated structure. If the user wants to send files (pictures, movies) from Windows to Ubuntu, it needs the help of extra software, which has been existed.

This is not the purpose of this assignment. But the structure for the socket is very simple. It only needs three conditions: an IP address, a port and a same Internet environment. And the socket can be used between different operating systems, like between Windows and Linux. Because of the characteristics of the simple structure and the transmission across different systems, socket is used as a connection between the PC application and the embedded software.

4.2.3 Establish socket

The establishment of the socket is reflected in developing the PC application and the embedded software. Before programming, it needs a clear view about the structure of the socket.

The socket is divided into two parts: the server part and the client part. In a socket communication, the client must input the IP address and the port of the server manually. Then the socket could be established. But according to the requirement, no extra operations are allowed on the platform. The connection part can only be done by the PC application. As a result, the PC application is a client and the embedded software is a server.

Figure 9 Structure of the socket

(27)

Figure 9 shows the structure of building a socket communication. To set up a socket, both the client part and the server part need to create a new socket with the code:

socket(). The usage of this code is the same in C# and C language. There are three parameters in this code: protocol family, socket type and the protocol. According to the introduction of the socket, it is made up of an IP address and a port. So the protocol family should be an address family, written as AF_INET. The type of the socket is TCP because the TCP transport protocol can guarantee that the data arrive at its destination (the platform) correctly without any duplication. So the second parameter is SOCK_STREAM standing for TCP. Since the type of the socket is TCP, as a result, the protocol also should be TCP written as IPPROTO_TCP.

Following the Figure 9, the code bind() can bundle the IP addresses and the port as an integer. Generally, the IP address is set as INADDR_ANY, which means any clients’ IP address can connect to the server. But for the port, there are about 65535 ports in a computer. It is difficult for the user to determine which port to use because some ports are unchangeable for some fixed uses like the 22^nd port is for the SSH connection, the 80^th port stands for HTTP protocol^○¹. So when programming, fix an idle port in the software for the establishment of the socket in case of the operator select a port, which has already been used. The 3333^rd port is used as a socket port in this programming, because this port is idle both in the computer and the platform. The code will be: bind (INADDR_ANY, 3333). socket() and bind() are the most important two steps in establishing a socket connection in the server part. Next, the server will start to listen to the instruction whether some clients want to connect. If there have the command, accept the client.

Turning to the client part (PC application), as long as the client gets the IP address and the port of the server, it can connect to the server after the client create a socket.

The connect() code contains the IP address of the server and the port (the 3333^rd port). After that, the server and the client can do some data exchange. The details about how to implement data exchange will be found in the next chapter.

(28)

4.2.4 Multiple platforms control

Now the prototype of the socket communication has been established. Next step is to meet the requirement of controlling more than one platform at the same time. This requirement can be realized when defining the socket in the client part (PC application) as an array. So for the PC application, the code for creating a socket in C# was written as: public Socket [] NewsSocket = new Socket[100]. Here the number 100 means the PC application can control at most 100 platforms at the same time.

Conclusion: Wi-Fi is selected as a communication between the PC and the platform.

And the socket is used to make the PC application and the embedded software communicate with each other. The PC application is the client and the embedded software is the server in the project. Following the structure of the socket, the prototype of the socket was set up by the code: socket(), bind(),listen() and accept() in C language and the code: socket() and connect() in C#. And the requirement of multiple platforms control is realized as well.

(29)

4.3 The PC application design

Following the methodology, next step is building the prototype. Chapter 4.3 introduces the design process of the PC application. According to the requirement in the company, the application should have a GUI for the users to operate. The first part of the chapter is an explanation of the GUI then the detail of the process will be introduced.

4.3.1 GUI design

The GUI design is separated into two tab pages. The first tab page is called

“Connection”, which is used to connect with the platform. The second one is a function page. The user can send files and control the platform to display the pictures or the movies on this page.

4.3.1.1 Connection page

Figure 10 shows the panel of the connection page. The port is set number 3333 in this project (The reason has been explained in chapter 4.2.3). So a text box was designed for the user to input the IP address of the platform. All the IP addresses will be listed in the list box. The operator needs to select the IP address that he wants and press connect button. The socket will be established. After finishing testing, press disconnect button to close the socket communication. The text “Status” was designed in order to make the user have a clear view about the condition of the communication.

It will show the word “Connect” or “Disconnect”, which can be found in the test phase.

Figure 10 Connection page

(30)

23

4.3.1.2 Function page

The function page includes three parts: sending files, showing the test pictures and playing the movie. The structure of this panel is shown in Figure 11.

Figure 11 Function page

First step is to design the part of sending files. In “Send file” area, the user can transfer the test pictures and movies to the platform. There are three steps for sending: click the “open file” button first, and then select a file. The name of the file will be shown in the “text box”. Finally click “send” button, the file will be sent to the platform.

Controlling the platform to display the pictures and the movie are two functions below the “send file”. If the number of the test pictures is more than one, add a text box that called “Duration” in the “Picture” area. The “Duration” represents how long one picture can last. Similarly, the button in “Movie” part contains all the foundational functions for playing a movie like Pause, Continue, Previous movie and Next movie.

(31)

4.3.2 Functionalities design

The designing of the GUI is finished. Besides the establishment of the socket, another two parts need be designed for the PC application: send files and send commands.

4.3.2.1 Send files to the platform

The function of transmitting files will enable as long as the user press the “Send”

button in the GUI. The flow chart gives a process of sending files to the platform that is shown in Figure 12.

(32)

So the path of the Linux should be defined manually. The string “linaro/receive/”

means all the files will be saved in a folder called “receive”. The word “linaro” equals to the symbol “C:\” in the Windows. Besides the path and the file name, the size of the file needs to be sent as well. A symbol “&” was added between the name and the file size. When the embedded software detects the “&” symbol, it stands for the path and name are finished receiving, the next data it received will be saved as the size of the file. The purpose of giving a size to the embedded software is to make a comparison with the real data the embedded software received. If they are identical, it means the software has received the data completely. To get the size of the file in C#, first step is to define a “File Stream” and put the file into this stream. Then the code:

Length.ToString() can measure the size of the file in the stream and save this size as a string.

In the socket communication, the code send() is used to transfer the data. There are three parameters in this code: a buffer, the length of the data and the socket flags. In the PC application, the letter “s”, path, file name, “&” and the size of the file will be put into a buffer called “msgHeadStr” as a complete string. The “send()” can only send byte type data. So before sending to the embedded software, the buffer will be changed from the string to the byte with the command: ”Encoding.Default.GetByte (msgHeadStr)”. The method of getting the length of the buffer is the same with getting size of the file. There are no different types of the data that need be sent in this project, so the flag will not be used and set the flag as 0.

Comparing with the CPU in the computer, the ability of processor of the platform is much weaker. If the size of file is too large, the processor cannot deal with all data in a short time, which may cause some data probably missing. So dividing the file into several parts and sending them in a loop can be a better solution to guarantee the integrity of the data. When programmed the application, the designer defined a default size called “max_buffer”. If the size of the file is larger than the “max_buffer”, it will go into a loop. Every time the size of the transmitted data is equal to the

“max_buffer” until all the data are finished sending.

4.3.2.2 Display the pictures and the movie

Next task of the PC application is to control the platform to display the test pictures and the movies. The solution of this function is to change the instructions into the data, and then send them to the server (embedded software). The data will be sent by send() code as well. When the server receives the commands, it will execute the operations. Table 2 lists the relationship between the instructions and the operations.

(33)

Table 2 Instructions and operations Instructions Operation the server will take

“R”+”G”+”B”+ duration Show the pictures and change the pictures according to the duration.

“S”+”T”+”O”+”P” Stop the pictures display.

“P”+”L”+”A”+”Y” Play the movie.

“P”+”A”+”U”+”S” Pause the movie.

“C”+”O”+”N”+”T” Continue to play the movie.

“N”+”E”+”X”+”T” Go to play the next movie.

“P”+”R”+”E”+”V” Go to play the previous movie.

“C”+”L”+”O”+”S” Close the movie player.

For example: when the user clicks the “Play” button in the “Movie” area in the GUI, the PC application will give the corresponding instruction “PLAY” to the embedded software. Then the embedded software will play a movie on the display.

Conclusion: The prototype of the PC application with a GUI is finished. The test of the PC application will cooperate with the embedded software in Chapter 4.5.

(34)

27

4.4 The embedded software design

The details of designing the embedded software in C language are written in this chapter. Like the PC application, there are two parts in the embedded software as well. They are: receiving files and executing the operations.

4.4.1 Receive the files

In the socket communication, the code recv() is used to get the data from the PC application. There are four parameters included in the recv(): the name of the created socket, the buffer for saving the data, the length of the data, and the flags. Here the length of the data means every time the maximum size of the data that the software can receive. So it should be the same value with the “max_buffer” defined in the PC application. To be consistent with the send() code, the flags should be set as 0.

Figure 13 shows the steps of receiving the file.

Figure 13 4 steps of receiving a file

(35)

The first step of data transmission is to send the information of the file (path, name and size) to the embedded software in the PC application. Similarly, the embedded software should receive these data following a structure as Figure 14 shown.

Figure 14 Design process of receiving the name and size

There are two loops in this structure. When the embedded software receives the first data: ‘s’, it means that the software needs to run the receive function. Because the buffer[0] stands for ‘s’, the data should be received from buffer[1] and save it into a

(36)

The embedded software will create an empty file according to the received path and the name. This part is achieved by fopen(). The code is completed with a pointer which points to the string: filename[] and the mode of the file. The mode is set to “ab+”

that means to give the software an authority to write the data into the empty file. The data will be written into the file with the command: fwrite() after the software receiving the data from the PC application. As the last step of receiving files, the parameters needs to fill in which includes the buffer, the data every time will be written, and the file created by fopen(). Because the data is sent in a loop, the last two steps will also be executed in a loop. fwrite() can return a value (mark: m) about how many data it real write into the file, the sum of the value makes a comparison with the file size in the filesize[]. If the sum is equal to the file size, the “receive” function is done.

4.4.2 Display the pictures and play the movie

According to the table of the instructions (in table 1), all the instructions are made up of 4 letters from buffer[0] to buffer[3]. So the operations will be acted before the embedded software checks the instructions in an “if” statement. For an instance:

If(((buffer[0] == ‘X’ && buffer[1] == ‘X’) && buffer[2] == ‘X’) && buffer[3] == ‘X’) (“X” represents a letter)

There are two applications called “feh” and “totem” in Ubuntu. “feh” is used as a picture viewer and “totem” is a media player application. These two applications are existed in Ubuntu when Ubuntu was installed on the development board. The users can input the name of the applications in the terminal in Ubuntu to start the software.

There is a code called “system()” that can simulate that situation.

When the embedded software receives a instruction from the PC application, it will use system() to show the pictures or play the movie. system() is easy to use, the designer needs to input the name of the software and some attached conditions in the code. For example: if a user wants to see the pictures in a full screen with 2 seconds duration, the instruction should be “feh -x -D 2”. The part: “-x -D 2” is the attached condition. The parameter “-x” means full screen mode and “-D” represents the duration mode, followed with a number. Then the complete code should be:

system(“feh -x -D 2”). The operator can see the pictures on the display in a full screen and they will be changed one by one in 2 seconds when this code is running in the software.

Conclusion: The prototype of the embedded software is finished established in C language. The detail of the code is in appendix II and the software will be tested in Chapter 4.5.

(37)

4.5 Test the communication and applications

The prototypes of the socket, the PC application and the embedded software have been built. Then it will go to the test phase. Because the configuration of DSI interface has not finished, the operator selected the High Definition Multimedia Interface (HDMI) output and displayed the image on a monitor to test the software. All the rest requirements are realized in this chapter.

4.5.1 Test the socket communication

To test whether the socket communication is established successfully or not, it needs the help of the GUI of the PC application. The operator input the IP address of the platform and press “Connect” button. The status displayed “Connect” as the Figure 15 shown. That means the PC application has connected to the embedded software with the successful establishment of the socket.

Figure 15 Socket communication statuses

However some mistakes might be produced like the operator inputs a wrong address, or the device does not connect to the Wi-Fi. These mistakes will cause a failure of the socket establishment. If the operator presses “Connect” button in an incorrect condition, the GUI will show a warning window to tell the user what is wrong. Figure 16 shows one of the situations, which will occur when the Wi-Fi signal is too poor to connect. It reminds the user to check the Wi-Fi connection.

Figure 16 Socket warning

(38)

4.5.2 Test the software

Two applications are used to do some data transmission, so the PC application and the embedded software will be tested together. The socket part of the software has been tested in Chapter 4.5.1, so this chapter focuses on testing the data transmission and controlling.

The operator chose a test picture on the PC and press “Send” button. After several seconds, he can see a message in a window (See Figure 17). This means the file has been sent to the platform correctly. Then the user can repeat his operations and send another file to the platform.

Figure 17 Send file successfully

In the test phase, the operator sent 10 pictures and two movies to the platform. All of them were finished with a “succeed” window. Then the operators selected 1 second in the duration and pressed “Play” button in the “Picture” area. The monitor immediately displayed all the pictures one by one. The successful implementation of this function indicates that the embedded software has received all the pictures without any data missing. Secondly, it also proved the PC application succeeded in giving a right instruction to the embedded software. Finally, the embedded software can execute the correct operations when received the instruction. Similarly, the movie is shown on the monitor successfully as well. And the two movies can switch to each other followed the button on the GUI.

The requirement of controlling more than one platform at the same time was tested as well. The designer used the PC application to control two platforms at the same time.

Both of the two platforms were able to receive the files and displayed the image on the monitor with the embedded software.

(39)

4.5.3 Problem and redesigning

Phenomenon: When the platform was displaying the pictures or the movies for more than 3 minutes, the operator cannot control the platform to stop displaying with the button on the GUI. And a warning that stated the socket connection was broken could be found on the PC.

Reason and solution: There is no data exchange between the PC application and the embedded software when the operator did not do any operations on the GUI. The socket communication will be broken if the socket is in “no data exchange” status for more than 3 minutes. The solution of keeping the socket communication in connecting status is to add a “Heartbeat”. The Heartbeat will be enabled automatically as long as the application finishes a function like sending a file, or sending an instruction. The principle of the “Heartbeat” is to make the PC application keep sending some data to the embedded software with “send()” command. The data is a symbol “!” in the project. The PC application will send the data to the embedded software once per 30 seconds, which is realized through a timer in C#. On the embedded software part, it needs to clean the buffer when it receives the symbol “!”.

Improvement: When the PC application does not send any data to the embedded software such as files, instructions, the “Heartbeat” will be enabled.

4.5.4 Setting up the platform

According to the requirement from the company, some operations such as connecting to the Wi-Fi, starting the embedded software should be finished automatically by the platform. So the designer needs to set up the platform before it is used. These settings were written into a reference manual. The user should follow the manual to set up the platform step by step. The manual can be found in Appendix IV.

Conclusion: In the test phase, the socket communication, the PC application and the embedded software were tested and performed well as expected. The socket is improved to make it more stable as well. And the last requirement is akso realized in this chapter.

(40)

33

4.6 Configuration of the DSI

Though the hardware of the DSI has been designed on the “expansion board”, the DSI driver should be configured in the Linux kernel. This chapter focuses on this part.

In this part, the designer should calculate the timing, modify the kernel with the calculated timing, compile the kernel and test it.

4.6.1 Display architecture of the platform

In the OMAP5432-EVM development board, there are two blocks, which make the DSI interface work: the display controller (called DISPC) and the DSI block as figure 18 shown.

Figure 18 Display architecture overview

The DSI1_A with a protocol engine is used in this project. As a part of the processor, the display controller generates the image data. The image data will be sent to the protocol engine of DSI1_A. The protocol engine splits the data into byte stream and then sends to the physical layer of the DSI interface. So in order to make the display controller wok with the DSI block precisely, the timings of the two blocks must be

(41)

calculated.

4.6.2 Timing calculation

This chapter lists all the parameters for calculating the timings and the values of a 1080*1920@60Hz LCD display as an example.

4.6.2.3 Input the parameters

In table 3, it lists all the basic but necessary parameters of the display. These parameters are used to calculate the timing of the display.

Table 3 Input for the calculation of timing parameters

Display parameters

Horizontal xres hfp,disp hsw,disp

hbp,disp +

H_total H_total= xres + hsa,disp + hfp,disp + hbp,disp

Vertical vres vfp,disp vsw,disp

vbp,disp +

V_total V_total = yres + vsa,disp + vfp,disp + vbp,disp

Parameters Unit

Bit Depth Bbp

NDL Lane

System parameters

F_{SYS_CLK} MHz

Followed the table 3, table 4 gives an explanation of the parameters

(42)

35 Table 4 Explanation of the parameter

Parameters Explanations

xres Resolution of the display in horizontal direction hfp,disp Horizontal Front Porch of the display

hsw,disp Horizontal Synchronize Pulse Width of the display hbp,disp Horizontal Back Porch of the display

yres Resolution of the display in vertical direction vfp,disp Vertical Front Porch of the display

vsw,disp Vertical Synchronize Pulse Width of the display vbp,disp Vertical Back Porch of the display

Bit Depth The number of bits per pixel

NDL The number of data lanes on the hardware

FSYS_CLK The frequency of the system clock

4.6.2.3 Determine the REGN, REGM and M4REG

Before the data is sent to the physical layer, it will goes to a control system called DSI Phase Lock Loop (PLL). It can keep the clock of the circuit output and the external reference clock in synchronous. The PLL will detect the change when the reference clock frequency changes and give a feedback to the system, to adjust output frequency, until both of them in synchronous again.

REGN and REGM are two factors in the DSI PLL, which needs to be calculated.

REGN is a “Reference clock divider” marked as “N” in the DSI PLL controller system.

And REGM means a “Feedback clock multiplier” written as “M”.M4REG is a “Divider value for clock source“, which determines the frequency of the pixel clock. All the relevant parameters are in table 2 and the formulas that can be found in the datasheet.

Then the values can be calculated following the formulas step by step. All the formulas are written in Appendix III.

(43)

4.5.2.4 Calculate the parameter

This chapter gives an example for the timing of the 1080*1920@60Hz LCD display First step is to fill the datasheet in table 3:

Display parameters

Horizontal

xres 1080 hfp,disp 74 hsw,disp 1 hbp,disp 49 +

Htotal 1204 Htotal = xres + hsa,disp + hfp,disp + hbp,disp

Vertical

vres 1920 vfp,disp 4 vsw,disp 2 hbp,disp 4 +

V_total 1930 V_total = yres + vsa,disp + vfp,disp + vbp,disp

Parameters Unit

Bit Depth 24 bbp

NDL 4 lanes

System parameters

F_{SYS_CLK} 38.4 MHz

Table 5 The parameters for the 1080*1920@60Hz DSI display Next step is to calculate the REGN, REGM and the M4REG:

REGN 32

REGM 639

M4REG 11

Pixel Clock 127800 Hz

After the timing has been calculated, next step should be modified the kernel with these values. But in the process of the project, it took a long time for programming and debugging. There was also a little delay in finding the program files, which need to be modified. These program files include the settings of the DSI interface for the

The Software for OMAP5 Platform Remote Control Tester

The Software for OMAP5 Platform Remote Control Tester

The Software for OMAP5 Platform Remote Control Tester

Final report

Summary

Foreword

Abbreviation

Table of Contents

1. Introduction

1.1 Company

1.2 Determined the quality of the LCD display

1.3 Assignment introduction

1.3.1 OMAP5432-EVM Development Board

1.3.2 OMAP5forLux

1.3.3 Assignment

1.4 Specific requirements

1.5 Questions of the assignment

2. Theoretical framework

2.1 Platform

2.1.1 Ubuntu

2.1.2 Android

2.1.3 Universal Serial Bus (USB) interface

2.2 Communication and data transmission

2.2.1 Wi-Fi communication

2.2.2 USB communication

2.2.3 Bluetooth communication

2.2.4 Socket

2.2.5 SSH

2.3 PC application on computer

2.3.1 Microsoft visual C Sharp

2.3.2 C# language

2.4 Embedded software for the platform

2.4.1 GNU Compiler Collection (GCC)

2.4.2 C language

2.5 Device under test

2.5.1 LCD Display

2.5.2 DSI interface

2.5.3 Linux Kernel

3. Methodology

3.1 Engineering Design Process

3.1.1 Introduction

3.1.2 Define the problem

3.1.3 Do background research

3.1.4 Specify requirements

3.1.5 Choose a best solution

3.1.6 Build the prototype, test and redesign

3.1.7 Communicate the result

4. Results

4.1 Operating system selection

4.2 Communication method

4.2.1 Communication method for the hardware

4.2.2 Communication for the software

4.2.3 Establish socket

4.2.4 Multiple platforms control

4.3 The PC application design

4.3.1 GUI design

4.3.2 Functionalities design

4.4 The embedded software design

4.4.1 Receive the files

4.4.2 Display the pictures and play the movie

4.5 Test the communication and applications

4.5.1 Test the socket communication

4.5.2 Test the software

4.5.3 Problem and redesigning

4.5.4 Setting up the platform

4.6 Configuration of the DSI

4.6.1 Display architecture of the platform

4.6.2 Timing calculation

Display parameters

Parameters Unit

System parameters

Parameters Explanations

Display parameters

Parameters Unit

System parameters