Web Automation in Python

Automation

Removing human dependency to perform a task and relying on electronic machines or robots is called automation.

Web Automation

Web automation enables the software or computer programs to perform the tasks automatically without human involvement. For instance, gathering valuable information from multiple web sources can be a hectic process, or submitting an assignment by clicking on multiple links can take several minutes for a human. Therefore, the developers can automate the processes according to their needs to perform the particular by pressing only limited keys and getting the work done.

Uses of web automation

Web automation can perform multiple tasks:

Delete emails such as spam or advertisement mails
Search the web to find helpful information
Fill out the forms such as lengthy google forms
Log into websites

Web automation in Python

The users can perform automation in python using valuable tools such as Selenium. Selenium is a powerful web tool to automate software tests, processes, and web scraping. This set of powerful libraries can help users connect the web with the program for developer-web interaction, enabling process automation. Moreover, it supports all the modern web browsers such as Firefox and Google Chrome, and the users can code it in multiple programming languages such as Python, C#, and Java. The steps below demonstrate Selenium’s installation and its usage in python to automate the logging process of some websites.

Installation and Setup of Selenium

Pip installs the Selenium package using the following command:

pip install selenium

The next step is to download the browser’s webdriver that matches the user’s browser version. It is essential to have the matching versions to use the webdriver for the web automation. The chocolatey users can install the chrome driver using the below command:

choco install chromedriver

If the webdriver is a zip file, extract it and move it to any directory of the user’s choice.

Using Selenium to log into a website

Create a new file, named “app.py”, and import the packages given below:

from selenium import webdriver
from selenium.webdriver.common.keys imports Keys

Set the path to the directory where the user has pasted the webdriver file.

PATH = “path to webdriver”
Driver = webdriver.Chrome(PATH)

Note: In this tutorial, we are using the chrome browser and its web driver. The users can replace these drivers and values with their own driver’s versions and browsers.

Write the code below in the app.py, which opens the web driver and requests it to open the particular website that the user has to automate.

driver.get(“the_website_url”)

Run the python file using the given command:

Python app.py

Running it will open the website on the browser. Right-click on the page to inspect it, and it will open a window containing all the elements used to create the particular web page. Find the id, class name, or path of the required fields.
The users can use one of the multiple methods to find the web page elements such as:
1. Find_Element_by_Id
2. Find_Elements_by_ClassName
3. Find_Elements_by_css_Selector
4. Find_Elements_by_xpath

The users need to find the password and username/email field element using the above methods.

Assign the password and email value to the python variables and use their values to send the keys to the requested website. Below is its coding demonstration:

email = “student@university.com”
password= “student123456”
email_var = driver.find_element_by_id(‘email-field-id’)
email_var.send_keys(email)
password_var = driver.find_element_by_id(‘password-field-id’)
password_var.send_keys(password)

Find the submit button’s id and use it to log in to the website successfully. The code automatically closes the browser after completing the login process and stops the program.

log_in = driver.find_element_by_id(‘login-button-id’)
log_in.click

Other useful articles: