Have you ever wondered how to enhance and ensure compatibility while developing your custom Android? As an individual developer, OEM, or vendor, you want to build excellent Android products without cutting ties with the Android ecosystem. Maintaining compatibility might be crucial, and the Android Open Source Project (AOSP) provides tools designed for this specific purpose.
In this article, we'll introduce the Android Vendor Test Suite (Android VTS), part of the xTS collection, discussing its features, functionality, and relevance of the Google vendor test suite.
What is VTS (Vendor Test Suite) in Android?
Android VTS is software the AOSP provides for OEMs, vendors, and manufacturers to test the vendor-system interfaces implemented in the OEM’s vendor image and Linux kernel. By focusing on the Vendor Interface (VINTF), Android VTS ensures compatibility and functionality of hardware components and vendor-specific customizations with Android.
OEMs and manufacturers need Android VTS
Starting with Android 9, AOSP modularized its source code to simplify manufacturing. They called this undertaking Project Treble. The reason for this change was that maintaining and updating existing devices and developing new ones has become increasingly complex and time-consuming. This is mainly due to the massive increase in devices available on the market since Androids' initial release.
Project Treble | Source: Own illustration
Before Android 9, there was a rather sequential process of updating devices. Chip manufacturers, framework distributors (e.g. AOSP, device vendors), and developers each had to wait for adjustments on the hardware implementation level.
Project Treble deserializes and breaks up these dependencies, allowing for a faster and more reliable development and update process. Device vendors can now make specific changes using a clear vendor interface (VINTF), requiring a robust test framework like Android VTS.
Android VTS originates from the Android compatibility test suite. It complements the Android CTS tests by supporting the decoupled manufacturing process of Android devices and acts as the standard test framework for the VINTF.
Android VTS tests multiple key parts of VINTF:
- Hardware Abstraction Layer (HAL)
- Common Kernel Interface (CKI)
- Vendor Native Development Kit (VNDK)
- Application Binary Interface (ABI)
Android VTS offers extensive test coverage at a lower level, ensuring integration and security, crucial for preventing exploits. Security on this level is essential since it is a target for exploits. This is why security features such as Android TEE exist.
Key test areas like HAL and CKI have dedicated test collections. You can write and extend tests using: Gtest, JUnit, Linux kernel, and Python3, enhancing Android VTS's value for development.
Working with Android VTS
Android VTS is part of the xTS test suites, like Android CTS and GTS Google. Each of them is testing for compliance with Google’s guidelines. However, developers can use these tools to build their independent products, it's not necessary to use the entire test library.
Adhering to the entire library is only mandatory if you aim for a Google Mobile Services (GMS) certification, ultimately getting Google’s proprietary services on your device.
Therefore, it makes sense to only use specific modules or subsets of the Android VTS test suite. Using this approach, you can benefit of Android VTS while keeping your efforts practical. Android VTS vendor test suite has great and comprehensive tests for various low-level component implementations.
Requirements for testing with Android VTS - Set up devices for VTS
After learning more about Android VTS, you might be curious about how to use it. VTS is part of the AOSP source code and requires no explicit download. However, some set up steps are necessary before testing.
Building a VTS-testable Android image
To run tests, first build a testable Android image. For the sake of this demonstration, we will go with a generic system image (GSI). First, set up a test environment using lunch. From the root of the source code directory, we run:
>> $ . build/envsetup.sh
>> $ lunch aosp_arm64-userdebug
These commands will build the GSI using the standard AOSP source code in the current directory. Next up, we need to compile the Android VTS package using the built-in unix command make:
>> $ make vts -j
This will build the VTS package and thus enable us to run Android VTS test cases.
Setting up test devices
Prepare your device for VTS Android tests by ensuring the following settings and prerequisites for your device under test (DUT):
1. Verify the following device settings:- Enable USB Debugging
- Enable Developer Mode
- (Host) Run ‘adb shell’ from the terminal
- (Host) Run ‘adb devices’ from the terminal
Android VTS is officially run on Linux, but can also be used on Windows with Python, adb, and Java.
Once the device is set up and installation is verified, run the Android VTS tests using the continuous Trade Federation (TF) test framework.
What is the Trade Federation (TF) continuous test framework in Android?
Trade Federation (TF) is a modular test framework that extends Android ADB with a recovery mechanisms. TF has built-in support for various test types, providing users the flexibility they need when testing multiple components. It comes with test scheduling and orchestrates multi-device testing to reduce testing time.
How to run a VTS test case?
To run Android VTS test cases, we first summon a vts shell using:
>> $ vts-tradefed
Within that shell, we can now run the provided test cases using:
>> $ run vts
This command runs the entire Android VTS suite, which takes time. However, if testing a subset is needed, Android VTS allows creating or using predefined test plans.
AOSP groups these plans into three groups:
- Official plans
These plans test Android VTS's core objectives and only include verified test cases, which are also mandatory to obtain Google’s licenses. - Staging plans
Staging plans are for continuous integration and contain experimental test cases with a dedicated prefix.
- Other plans
These are special-purpose plans developers may find useful.
Using an existing plan allows in-depth tests for a specific component, such as running an Android VTS plan for HAL using:
>> $ run vts-hal
While this helps reduce the overall test time, narrowing tests further is helpful to streamline the development processes. For this reason, we can use test modules, which are subsets of Android VTS test plans. However, no curated list exists, so you will likely find yourself digging into the source code.
Beyond compatibility: Using Android VTS to build great products
Android VTS is an essential part of the Android development pipeline. Its primary focus is the integration between hardware-dependent and hardware-independent code while speeding up time-to-market for updates, and new products releases.
The tool is beneficial for development. As a flexible and customizable test framework, it can help to create custom AOSP ROMs and products beyond Androids original intent. It ensures efficient testing of customizations and compatibility with CTS VTS Android and AOSP standards.
With years of expertise customizing Android, emteria can let your product vision become reality. We can provide the Android OS foundation for amazing products, from ready-to-use to fully customized solutions. Contact us today to learn more!
Build secure Android products, keep them up-to-date
See why emteria is the chosen Android™ customization & management platform for product builders — build modern Android products with the highest security standards.
Table of contents
