Getting Started With Appium in AWS Device Farm for iOS

Getting Started With Appium in AWS Device Farm for iOS
Photo by Kelly Sikkema / Unsplash

Lately I've been interested in moving our iOS app testing to AWS Device Farm. This allows us to share testing results across different teams, view historical performance, and easily test on many different iOS devices. I thought I would share my experience here and how I got started.

AWS Device Farm currently supports Appium tests for Java JUnit, Java TestNG, and Python. I wrote my tests in Python so this guide will reflect that.

Before you get started, you'll want to either build and archive your iOS app (.ipa file) or have your dev team provide it to you. Make sure you also have Xcode installed and up-to-date.

Preparing your tests

  1. I recommend installing the Appium GUI. This will really help in finding element names and getting the syntax down when you first start. As I said above, I wrote my tests in Python so if you take that approach go ahead and install the Appium Python Client.

    pip install Appium-Python-Client

  2. Launch Appium, click the Apple icon and select the path to where your app is located. Choose a specific device you want to test on.

  3. Click "Launch" and once the server is running, click the magnifying glass icon to open the Inspector. At this point, your app should open in a simulator and the Appium inspector will open in a separate window once your app is completely loaded.

  4. If you're unfamiliar with the syntax, I recommend clicking Record and navigating through elements in your app. Elements will be highlighted in red in a screenshot on the right. Sending a tap or keystrokes to an element will execute the action and translate it to code in the window below. You can also toggle between languages (again, I'm using Python).

  5. Once I had a pretty good idea of the syntax I needed to use and what I needed to accomplish for my test, I just wrote up my tests in a text editor and used the Appium inspector to find the XPath or element name I needed to reference.

  6. AWS Labs has posted a sample Device Farm test on their Gitlab page that can be found here which I used as a guide for my formatting. Here's a truncated sample of my Python script:

     import os
 import unittest
 from appium import webdriver
 from time import sleep

 class DeviceFarmAppiumiOSTests(unittest.TestCase):
     def setUp(self):
         desired_caps = {}
         desired_caps['appium-version'] = '1.0'
         desired_caps['platformName'] = 'iOS'
         desired_caps['platformVersion'] = '9.3'
         desired_caps['deviceName'] = 'iPad Air'
         desired_caps['app'] = os.path.abspath('/your/local/app/') #replace with your local app path for testing the script locally
         self.driver = webdriver.Remote('http://0.0.0.0:4723/wd/hub', desired_caps)

    def test_devicefarm(self):
         self.driver.find_element_by_xpath("//UIAApplication[1]/UIAWindow[1]/UIATextField[1]").send_keys("iostest")
         self.driver.find_element_by_xpath("//UIAApplication[1]/UIAWindow[1]/UIASecureTextField[1]").send_keys("password")
         screenshot_folder = os.getenv('SCREENSHOT_PATH', '/tmp')
         self.driver.save_screenshot(screenshot_folder + '/loginscreen.png')
         self.driver.find_element_by_name("loginBtn").click()
         self.driver.save_screenshot(screenshot_folder + '/homepage.png')
         self.driver.find_element_by_name("logoutBtn").click()
         self.driver.save_screenshot(screenshot_folder + '/logout.png')

 if __name__ == '__main__':
     suite = unittest.TestLoader().loadTestsFromTestCase(DeviceFarmAppiumiOSTests)
     unittest.TextTestRunner(verbosity=2).run(suite)

As you can see, I've also chosen a few spots in between actions where I'm going to have AWS Device Farm take a screenshot. These images get saved in your runs and can be helpful in troubleshooting later on.

  1. Create a virtual environment using Python virtualenv

     $ virtualenv workspace
 $ cd workspace
 $ source bin/activate
 $ pip install pytest

  2. Put your python test script in the tests/ folder

  3. Run py.test to confirm that your script is discovered and can be run

     $ py.test --collect-only tests/

You should recive an output similar to this:

    ============================= test session starts ==============================
    platform darwin -- Python 2.7.10, pytest-2.9.1, py-1.4.31, pluggy-0.3.1
    rootdir: /Users/USERNAME/Desktop/workspace/tests, inifile: 
    collected 1 items 
    <Module 'test_unittest.py'>
      <UnitTestCase 'DeviceFarmAppiumiOSTests'>
        <TestCaseFunction 'test_devicefarm'>
    
    ========================= no tests ran in 0.05 seconds =========================

  1. Generate your requirements.txt file

    $ pip freeze > requirements.txt

  2. In the same directory, run the following to create a wheelhouse directory

    $ pip wheel --wheel-dir wheelhouse -r requirements.txt

  3. Clean up the cached files using:

    $ find . -name '__pycache__' -type d -exec rm -r {} +
$ find . -name '*.pyc' -exec rm -f {} +
$ find . -name '*.pyo' -exec rm -f {} +
$ find . -name '*~' -exec rm -f {} +

  4. Zip your tests/ folder, wheelhouse/ folder, and requirements.txt into a single archive for upload to Device Farm

    $ zip -r test_bundle.zip tests/ wheelhouse/ requirements.txt

Running your tests on Device Farm

Now that your tests are ready for upload, let's take a look at how easy it is to start running your tests on Amazon's cloud.

  1. Navigate to the Device Farm service in AWS and click "Get Started" if this is your first time using it. Click "Create a new project" and give your project a name.

  2. Inside your new project, click "Create a new run". Click the Android/iOS symbol if it isn't already selected and upload your .ipa file. Once uploaded, the run name will auto-populate with the name of your .ipa file. Go ahead and rename it if you'd like then click Next.

  3. Under the "Configure a test" page, click the Appium Python radio button then upload your test_bundle.zip we created earlier. If you did everything right so far, it should process for a moment then give you a green checkmark. Click Next.

  4. On the next page you'll choose your device pool to test against. By default, the top 5 devices are pre-selected. For my first test, I created a new device pool that only contained a single device. Click "Create a new pool" and select as many or as few devices you wish to test against. Give your new device pool a name and description. Click Next.

  5. If you need to modify the device state in any way, you can do so on this page. There's an option to upload another app in case your run is dependent on the existence of another. You can change the device location and the locale as well. (There's also an option to upload a zip file to "Add extra data". I have no idea what this is or how it works, so if anyone is able to provide some insight here in the comments I'd greatly appreciate it!)

  6. Click "Review and start run". Confirm your settings then click "Confirm and start run". Once your run is complete, you'll get to see whether it passed or failed, screenshots, performance, and collect several logs.

I'm still very much a newbie to AWS Device Farm so if you have any questions, I'll attempt to answer to the best of my ability. And if you have any comments that might further my knowledge of this service, even better! Good luck!

Mastodon