Introduction to Sikuli

Sikuli is a scripting language which helps us to automate Software testing of Graphical User Interface(GUI). Sikuli is open source automation testing tool that is developed in java. Using Sikuli Automation tool, we can automate whatever we see on the screen. It basically uses image recognition technology as it helps us to identify and control GUI components. It is robust and powerful GUI automation tool. It can be used to verify screenshots and PowerPoint slides as well.

Sikuli can be used to Automate “Desktop, Window, Adobe/Audio/Video/players, Flash, Flex objects, Games Website objects, Windows Applications and more”.

It can be used in the scenarios where applications are very complex but have very simple screens or visualization.

It is very useful in cases in which the application code gets changed frequently but GUI components remain the same.

Using Sikuli, we can automate both Web and Windows based applications. Key features of Sikuli Tool

• GUI Automation testing tool.
• Follows Image Recognition technology. It provides simple API e. all methods can be accessed using screen class object.
• It is useful when there is no easy access to a GUI’s internal or source code.
• It is best for use on Flash applications.
• It is very useful in cases in which the application code gets changed frequently but GUI components remain the same. Easy to unit test at low level and object level.
• Easy to learn and easy to use
• Easy to integrate Sikuli with selenium.

Basic commands and features of Sikuli 1 .Find It is automation approach that helps us to find the exact pattern on the screen or image. This function locates particular GUI element to interact with. This hybrid method uses template matching for finding small patterns and invariant feature voting for finding large patterns. It takes a visual pattern that specifies the element appearance and searches the whole screen or part of the screen and returns regions matching this pattern or false if no such region can be found.

Region is basically the area that we select on screen. Example: – b=find () We can use variable b to store the region and then can call ‘wait’ or ‘click’ method so that it will continue running the script within that area. Selecting a region and assigning it to a variable also helps when there are multiple similar items on screen and we want to deal with one at a time. b.click () , b.wait () . 2. Wait() Wait() is used to slow down the script to let the currently running process finish or to make something vanish. We have optional parameter here which can be either seconds or minutes which will wait until something happens. 3. Pattern It is Image based pattern object for tuning and specific matching.

We can have exact matching that can be done by search method that is pixel by pixel. We have similar() that helps us to find the similarity between 0 and 1 threshold that will define how similar matching regions are. 4.Action The Action command specifies what keyword and/or mouse events to be issued to the center of a region found by find ().

We can perform click , double click, drag and drop functions. 5.Highlight This is another basic command used to draw a box around a region. 6.Visual Representation A visual representation is a kind of visual dictionary. It is a data type used for storing key-value pairs using images as keys. It provides Sikuli Script with a convenient programming interface to access Sikuli search core functionality.

Using a visual dictionary, a user can easily automate the tasks of saving and retrieving data based on Images. Script Editor

• Take a Screenshot: – It helps to take the screenshot of any GUI element to add to a script, a user can click on the take screenshot button in the toolbar to enter the screen capture mode. The editor hides itself automatically to reveal the desktop underneath and the user can draw a rectangle around an element to capture its screenshot. The captured image can be embedded in any statement and displayed as an inline image.
• Load an image from Disk: – The user can load an existing image file from disk or type the filename or URL of an image and the editor automatically loads it and displays it as a thumbnail.
• Select a Region: – User can also specify the arbitrary region of the screen to confine the search to that region.
• Execute a script: – User can click on execute button and then at that time editor will be hidden and script will be executed.
• Script editor: – Script editor is a placeholder where we write code.
• Console: – Console is a placeholder where we get error messages that gets triggered if code is not run properly.

Basic program for understanding of selenium with Sikuli: – import java.util.concurrent.TimeUnit;

import org.openqa.selenium.By;

import org.openqa.selenium.WebDriver;

import org.openqa.selenium.firefox.FirefoxDriver;

import org.sikuli.script.FindFailed;

import org.sikuli.script.Screen;

import org.testng.Assert;

import org.testng.annotations.Test;

public class SikuliTest {

@Test

public void functionName() throws FindFailed {

//Initialize Firefox Driver
WebDriver driver = new FirefoxDriver();

//Launch Amazon website
driver.get(“http://www.amazon.com/”);

//Maximize the browser
driver.manage().window().maximize();

//Apply implicit wait on driver instance
driver.manage().timeouts().implicitlyWait(35, TimeUnit.SECONDS);

//Create an object of Screen class
Screen scr = new Screen();

//Perform click on drop down list
scr.click(“C:/Users/workspace/SikuliProject/Allcategories.png”);

//Click on Bag option
scr.click(“C:/Users/workspace/SikuliProject/BagsOptions.png”);

//Click on Search button
scr.click(“C:/Users/workspace/SikuliProject/SearchButton.png”);

//Fetch the heading from the web page
String text = driver.findElement(By.tagName(“h1”)).getText();
System.out.println(text);
Assert.assertTrue(text.equalsIgnoreCase(“Bags”), “Bags page is not opened”);

}

}