Extract and print all the URL links in a website

Sometimes when we are testing web sites we need check how many active URLs are available, May be per user, per access group level.  So lets see how we can do that .  

Extract All the URLs.

WebDriver A = new FirefoxDriver(); 

A.get("http://www.google.com"); 

//Get all the URLs and create a list 

List<WebElement>URLS2 = A.findElements(By.xpath("//a[(@href)]"));

//Get the list size (URL Size)

int count2 = URLS2.size(); 

//Prints the list size 

System.out.println(count2);

//Access each list element and get the URL in a loop

for(WebElement a : URLS2) {

String link = a.getAttribute("href"); 

System.out.println(link);

   }

Above code will load the google page and will extract all the available URLs to a List. In java Lists are type of a data structure where we can store different set of data. 

List<WebElement>URLS2 = A.findElements(By.xpath("//a[(@href)]"));

In this line we are creating a List named "URLS2" and assigning all the "@href" links to it. I hope you are familiarize with what meant by  "xpath("//a[(@href)]")" else you can refer my blog post on "Xpath". 

when we assign the URL values to the list we can access them one by one. The loop does it. we pass the list size parameter and prints the URL values one by one.  the out put of this code will be like below 

Ill be posting how the dead links can be checked in future.