At Wingify, we believe in open source and actively seek opportunities to give back to the community. We make use of a lot of open source technologies and libraries in our day to day work. And to share back the love, we have open sourced quite a few things over the past.
Something really cool that came into picture from his infographics was that Wingify was leading other organizations in terms of the most starred repository - our Please.js. Moreover, we have three more of our repositories that made it to the top 10!
We, at Wingify, have been writing e2e test cases for our A/B testing app for the past 5 months using protractor. Writing e2e scripts is easy but making them work 90% of the times on all browsers makes you go nuts! Sometimes, I feel that the browser is moody, but of course, we have to deal with all its moods and make sure that test cases are robust.
Getting your e2e tests to run smoothly everytime comes with experience. And, learning from my experiences testing our app, here is a post describing them.
An Overview of Protractor
Protractor is a specially designed wrapper around WebDriverJs to automate e2e testing for angular apps. The following figure gives an overview of the architecture involved for automation and testing :
Once the setup is done, you are ready to write and run e2e scripts. Next you need a report of failed and passed test cases along with screenshots. protractor-html-screenshot-reporter , an npm module, provides you with an Html report of the test cases along with screenshots. Set it up as mentioned in npm documentation
Test Scenario and Test Data follow a one-to-many relationship, so we can’t hardcode the test data within the code. For example: The same login test script can be used to test the login of different types of users. To make life easy, it is a better option to make use of .xls files (Microsoft Excel, OpenOffice etc) and import data dynamically into the script, while execution.
xlsjs npm module lets us fetch the data from an .xls file and use it in script. Follow the below mentioned steps to set it up :
Call function as:
Test both Angular.js and non-Angular.js based pages
Our use case involves using Protractor for an Angular.js based app, but it works pretty well for non-Angular.js pages as well.
Simply set the following flag to true and access the webdriver instance using browser.driver instead of element as shown below :
For instance, the following code for an angular page:
would be written as shown below for non angular page:
Sounds cool? Now let’s dig deeper in the protractor world.
A piece of protractor code
Before we start with the advance functions, let’s have a look at a simple login test case, where we verify that the user should be redirected to a welcome page after login.
Let’s study the code:
Describe , it and expect are jasmine framework methods to write the tests easily. Read more about them here.
beforeEach function in the code above executes before all it blocks, however, you might not want to execute the code in beforeEach for all it blocks.
Tip: To control its execution you can use a flag variable as shown in the code below.
Multiple ways to select elements:
Interacting with the DOM:
element: returns a single element
element.all: returns a collection of elements. Use get(index), first(), and last() functions to get a single element out of the collection.
Other awesome functions
filter : Similar to get(index), first() and last() functions, filter takes a collection of elements and returns a single element. The only difference is that the element can be selected based on the specified condition. This is useful when there is a dynamic/long list of elements having same selector path and you need to get an element using its text or any unique property.
Let’s take this example of clicking on a date from a calendar.
All the dates elements have the same selector. Therefore, define a filter function as :
Now to click on date 17, simply call the function as :
each : Use it when the same action has to be taken for all elements having common selector. For example : clearing the list of input fields in a Signup form.
map : Mapping a collection of elements in an array without use of protractor map function involves a lot of code to deal with all the proimises one by one. On the contrary, using map function for the same purpose is a piece of cake. Map function iterates through each element found with the locator and then resolves all the promises to return a promise with an array of values. For example : To get the text of all elements (with ng-repeat = ‘option in Options’) in an array, write the code as :
Tips and tricks
Manage Browser logs: There will always be certain scenarios which would not be covered in e2e scripts. Therefore, it is a smart move to always check browser console errors for any unexpected issue in the app. The following piece of code allows you to keep a check at browser logs and fails the test cases if there are any errors :
Combine element statements to move around the dom : Xpath provides an excellent way to move up and down the dom.
Common UseCase: Error messages are often displayed as a sibling to input or submit types. Therefore, instead of using a different selector path for error message, xpath can be used to pick the sibling.
To verify the error message “invalid URL”, simply write the assertion as:
Never use protractor element statements inside loop: The simple reason is that the webdriverJS (protractor) API is asynchronous. Element statements returns a promise and that promise is in unresolved state while the code below the statements continues to execute. This leads to unpredictable results. Hence, it is advisable to use recursive functions instead of loops.
Debug the tests using elementexplorer.js: elementexplorer.js lets you test the page interactively. You will find this JS file in node_modules/protractor/bin directory. Start the selenium server and run command:
Browser will load the URL and you will see > prompt. Use browser, element and protractor variables to interact with page.
Note: Make sure that the developer tools are closed while running commands in elementexplorer.js prompt, otherwise you will face an unexpected error as “TypeError: Cannot read property ‘click’ of null”
Maintaining and reusing test cases
Let’s admit it, e2e test cases are not easy to be maintained and updated. You have to organize the tests in a way that they can be edited easily. Let’s have a glance at Page-Objects, common-files and tests approach that we follow at Wingify, using nodeJS require and exports functions.
page-objects: Page-objects is a commonly used practice across the industry while writing e2e tests. It enables you to write clean tests by listing all the information about the elements in a page-object file. This means that you only need to change the page object file, in case of any change in template of app.
Our Application has more than 50 screens. Therefore we list all the page-objects i.e. dom elements of each screen in a seprate JS file. Take a look at login screen page-object file :
common-module: The idea is to divide the entire e2e scenario in small reusable functions in a way that these functions can be used in other e2e scenarios as well. These reusable functions can be grouped in different files for maintainblity. The login and logout module is used in many e2e scenarios. So, both can be clubbed in a file as shown below :
e2e-scripts: Include all the common-module functions to write the complete e2e-script as shown below :
Take an extra step to write test cases in a way that they can be edited and maintained with ease.
Create a modular approach keeping scalability in mind.
Dive deep in the protractor world, use protractor’s awesome functions and have fun!!
Hope this post was a good enough reference to help you write end-to-end tests in a better way. If things might be unclear, or you have any questions, let us know via comments.
To begin with, lets talk about two of the most important things are that come to mind when we talk about performance testing.
The Metrics to Measure
First lets consider the metrics to measure. Few important metrics that should always be considered are:
Number of HTTP Request and HTTP Response status.
Coming to second part i.e. Dependencies. Now this could be broadly classified by 2 groups:
Server-side (API level) testing
In single-page apps, the performance equally depends on both the client-side and the server-side.
Client-side performance testing can also be done using popular tools like Google Page Speed or Webpagetest.org. But they cannot test different modules of the application separately. They’d just test the URLs you enter. To test different sections of your application, we can follow a different approach.
In this blog post, I’m going to show you how to use the most popular open-source tool (JMeter)[http://jmeter.apache.org/] to performance test AJAX-enabled websites.
Now to overcome this challenge we have a few options we can work on:
Use WebDriver Sampler to measure the response time in a real time browser. Combining this with JMeter load test, we can measure the real time user experience when we apply severe load.
Use JUnit Sampler to create selenium scripts using tools like Eclipse. Using this approach one can export JAR fie to JMeter and run the test in browser.
Simulate an Ajax request using JSR223 sampler.
Lets talk in further detail about these methods to performance test.
Using JMeter WebDriver Sampler with Selenium
Web Driver Sampler automates the execution and collection of Performance metrics on the Browser (client-side). You can download this plugin from the link shared below.
A large part of performance testing, up to this point, has been on the server side of things. However, with the advancement of technology, HTML5, JS and CSS improvements, more and more logic and behaviour have been pushed down to the client. This adds to the overall perceived performance of website/webapp, but this metric is not available in JMeter.
Simply add the following to your test plan:
Firefox Driver Config
Web Driver Sampler
View Results Table
The only problem with approach is that the automation capability is limited. But again that depends on the application.
Using this method all we need to do is create a JAR file using Eclipse and export it to JMeter.
Creating JAR using Eclipse
Create a JUnit Test case in your project.
Write the following selenium code to open your homepage.
Note: In the above code we are using RemoteWebDriver to open the browser on a different machine since we will integrate our test with jenkins.
Once this is done you just need to export the JUnit test case to JMeter. Just copy the JAR file into JMeter/extras/JUnit folder and restart JMeter.
After this just click on “Search for JUnit 4 annotations” in case you created a JUnit 4 test case and you ll find the JAR file with class name in drop down.
Simulate an Ajax request using JSR223 sampler
Normally this method is not recommended. But if above solutions don’t work well with your system you can go for Beanshell and use any scripting language like groovy to create Ajax request.
Below are some points to keep in mind while creating performance test plan:
Create custom scripts for different use cases and create a threshold for various metrics.
You can also use headless browser testing using HtmlUnitDriver or Xvfb depending on your system. This approach would work well if you need to combine these with your load test.
You can integrate your tests with Jenkins using the performance plugin. It really helps with reporting and you can simply run it with each new build. In fact, that is also how we do it here at Wingify.
Client-Side performance testing is always done using 1 or 2 threads. But if you need to use it with your load test then you can simulate as many threads as you wish to.
Back in November, I, along with some colleagues from Wingify went to Singapore to attend CSSConf and JSConf Asia. A part of DevFest Asia, it is the best way to meet and connect with front-end designers and developers in South East Asia. It was the first time CSSConf was happening in Asia, and our own Kushagra Gour had the opportunity to speak at it. He talked about 10 Commandments for efficient CSS architecture. The conference venue was Hotel Amara Sanctuary, on the island of Sentosa, Singapore.
The day began with boarding the MRT and the Sentosa Express to get to the venue, and meanwhile being mesmerized by the marvelous Singaporean skyscrapers. A sip of coffee at the Jimmy Monkey Cafe made our eyes open wide, and began Thomas’ introduction to Asia’s first CSSConf. It is commendable for a small bunch of people to organize conferences for the community at such large scale, working part time.
The quality of the talks at the conference was way above our expectations for a first-time event. The speakers and the topics they spoke about were pretty interesting. Below are the talks that I found really intriguing, and I recommend everyone to go through the videos and slides for them:
Her talk made an interesting point: with all the rapid development happening in the front-end realm, we’re making use of tools more than ever, which when mixed with biases and preferences, do more harm than good. She explained what puts apart good tools from the bad ones, and the telltale signs of a tool doing more harm than good.
As front-end developers, we have been conditioned to live and breathe CSS, both good and bad parts, beckoning a love-hate relationship with it. But it is rather cumbersome for someone new coming in ease into the concept of cascading style sheets, and that coupled with misused practices makes people think of CSS as some sort of dark magic. Christian’s talk separates the good parts from the bad ones, and points the listeners into a direction everyone can take to make CSS a language everyone can love.
AirBnB’s product is so simple, yet efficient to use, especially their mobile app that it makes you feel a sense of satisfaction competing products or even luxurious hotels couldn’t match. Booking for a stay with the locals via AirBnB has always been exciting inspiring for me. And today it was a fortunate time to see the face behind such great product design.
If you’re like me, perhaps you ask yourself this question as the sun rises each morning. Recall a time when that wasn’t really the question? Ben’s talk took the listeners on a nostalgic ride to the years of hacking around and creating things from the past. Recall spending nights working on creating things as the first ray of sun hits the window, the pride of showing around your project in your peer group, hours upon hours of creative brainstorming to build something exciting? Ben extracts out the secret sauce for that feeling of happiness that stems from creating something and how to hold on to it to keep treading that road.
Words of Wisdom
JSConf had multitude of interesting talks. Below are the ones I found rather inspiring to attend:
“Your password must contain an uppercase and a lowercase letter, a number, a dinosaur’s name and your grandmother’s maiden name”: do sights of such messages make you cringe? Or perhaps you recall the frustration of filling out a large form and you missed out checking those terms and conditions and the page complains by asking you to fill everything over again? If you do, this talk will let you share your frustrations and learn about what you as a developer can do to create better forms for a better web.
Tingan goes about solving the problem of i18n and l10n by introducing concepts like CLDR, pluralization, and ICU’s message formats. He also publishes his localization module l10ns at the end of his talk. If you’re planning to translate your app into multiple languages, especially east-Asian ones, this is a must see talk.
BitTorrent is a great protocol to share data between users across the world. Feross introduces the underlying functionality of BitTorrent and the advantages of a distributed network when compared to a centralized one. He also highlights certain shortcomings of the protocol that are centralized, like torrent files and tracker servers, and how we could solve these problems. Finally he describes WebTorrent, a project that makes use of WebRTC and special “hybrid clients” to connect to the BitTorrent network.
It might seem like I’ve described every talk at both the events, but the truth is that the quality of the talks was so impressive that it is hard to not praise the speakers for the efforts they have put in to prepare a stellar presentation. Looking forward to the next installment of DevFest Asia in 2015!
Last week, we announced that Wingify would be sponsoring the JSFoo 2014 conference in Bangalore. We have always been looking out for opportunities to give back to the community, and what could have been better than hosting a run-up event for JSFoo in Delhi. We got in touch with HasGeek and expressed interest in hosting the Delhi run-up event. They were more than happy to let us do so.
Since it was the first time we hosted such a big event, we were both excited and nervous. We had started preparing for the event about a month in advance. To market the event, we spread out information about the event on meetup groups we were connected with, and also created a Facebook event for it. In total, we received about 100-120 signups for the event. Of all the signups, we expected about one-third of them to actually turn up for the event, for whom we started to arrange for food and logistics.
What’s exciting is that more than 50 external guests from 30 different companies and colleges showed up. We were really humbled with such great turnout and would like to thank each one of you who joined us for the event.
Kushagra Gour taking workshop on “Learning JS concepts/techniques by implementing jQuery” with his awesome "Prayas" system on screen
The workshop was followed by lunch after which we had Shwetank Dixit talking on “Understanding WebRTC: A Whirlwind Tour”. Shwetank introduced WebRTC and then went in-depth on the topic, explaining how the technology works and how are different network related problems addressed. He also showed some really cool WebRTC demos. Appear.in was one of them. We hope that attendees would have gone back and tried WebRTC.
Shwetank giving demo of WebRTC
Next was our very own Himanshu Kapoor talking on “Managing API Resources and Their Relationships on the Front-end“. This was indeed a very interesting talk wherein Himanshu talked about how we handle RESTFul resources on the front-end. The talk was primarily about our learnings of maintaining API resources and their relationships with using our internal project called ResourceManager for AngularJS (soon to be open sourced).
The day ended on a high note with everyone really appreciating the speakers and the organizers for all the hard work everyone put in. There were a lot of discussion after the event on organizing more such events for different communities in Delhi NCR region. Wingify as a company believes in engaging and giving back to the community and we would be more than happy to host more such events in future. If you feel that there is any technology related community which needs a place to host the event, please write to us at [email protected].