Java URLconnection | HttpURLConnection Example

We are going to see how to deal with URLs in Java in this chapter. We will also explore the URL class, Java URLConnection Class, HttpURLConnection and also see some methods of Java InetAddress Class. While exploring these we will see some cool examples too.

Before we begin it is important that you know what URL is.

Understanding URL

URL is an acronym for Uniform Resource Locator. And you thought it was just one word, huh? It tells you about it through the moniker itself.

It is pointing to a resource on the internet. For example:

http://dumbitdude.com

It is a URL pointing to the home page.

The first part of the URL “http” is a protocol. The second part “dumbitdude.com” is the server name or the host name. In actuality there must be some address (IP address) corresponding to that which has been simply mapped against the name of the website. The unique phrase before .com, here dumbitdude is also called as domain.

In some cases a URL ends with a file name. For example

http://dumbitdude/socket-programming-in-java

Here “/socket-programming-in-java” would be the file name.

Sometimes you find a port number lurking there as well like this:

http://dumbitdude.com:80

Here 80 is the port number. If there is no port no. trying to retrieve it using one of the methods of URL class will return -1.

URL Class

There is a class available in Java known as URL class, that makes grabbing inputs from a URL a piece of cake. Like any class in Java, it has its very own set of Constructors and methods. Here have a look at them.

The following constructors of URL Class can be used:

• URL(String spec)
• URL(String protocol, String host, int port, String file)
• URL(String protocol, String host, int port, String file, URLStreamHandler handler)
• URL(String protocol, String host, String file)
• URL(URL context, String spec)
• URL(URL context, String spec, URLStreamHandler handler)

There are tons of methods available in URL class. Here they have been lined up below:

1. equals(Object obj)
2. getAuthority()
3. getContent()
4. getContent(Class[] classes)
5. getDefaultPort()
6. getPath()
7. getQuery()
8. getRef()
9. getUserInfo()
10. hashCode()
11. getProtocol()
12. getHost()
13. getFile()
14. getPort()
15. openConnection()
16. openConnection(Proxy proxy)
17. openStream()
18. sameFile(URL other)
19. set(String protocol, String host, int port, String file, String ref)
20. set(String protocol, String host, int port, String authority, String userInfo, String path, String query, String ref)
21. setURLStreamHandlerFactory(URLStreamHandlerFactory fac)
22. toExternalForm()
23. toString()
24. toURI()

The method openConnection returns a URLConnection instance while openStream() returns an InputStream.

There’s a sameFile() method that can be used to compare two URLs as well. It returns boolean.

Examples for URL Class

Let’s see some examples. It would clear some things up right away:

You need to use the try-catch block to handle MalformedURLException received whilst using the URL constructor:

 try { 

URL url = new URL("http://dumbitdude.com/socket-programming-in-java"); 

System.out.println("The protocol is: " +url.getProtocol()); 
System.out.println("The file is: " + url.getFile()); 
System.out.println("The host name is: " + url.getHost()); 
System.out.println("The port no. is: " + url.getPort()); } 

catch (MalformedURLException e) { 
e.printStackTrace(); 
}

If you run the above program you will get:

The protocol is: http
The file is: /socket-programming-in-java
The host name is: dumbitdude.com
The port no. is: -1

As you can see there was no port no. involved and hence we get -1.

When you have a query in your URL you can grab the query part using the getQuery() method like this:

URL url = new URL("https://www.google.co.in/?gfe_rd=cr&ei=wWNdWc2zMefy8AfXrpugCA");

System.out.println(url.getQuery());

Searching the above will give you:

gfe_rd=cr&ei=wWNdWc2zMefy8AfXrpugCA

Second Example

Here’s a much better example that intends to cover most of the things that we can obtain:

try {

URL url = new URL("http://username:password@example.com:123/path/data?key=value&key2=value2#fragid1");

System.out.println("The protocol is: " +url.getProtocol()); 
System.out.println("The file is: " + url.getFile()); 
System.out.println("The host name is: " + url.getHost()); 
System.out.println("The port no. is: " + url.getPort()); 
System.out.println("The Query is: " + url.getQuery()); 
System.out.println("The Path is: " + url.getPath()); 
System.out.println("The User info is: "+ url.getUserInfo()); 
System.out.println("The Reference is: " + url.getRef()); 
System.out.println("The Authority is: " + url.getAuthority()); 
System.out.println("The URI format is: " + url.toURI()); 
System.out.println("The String representation is: " + url.toExternalForm());
} catch (MalformedURLException e) { 
e.printStackTrace(); 
} catch (URISyntaxException u) { 
u.printStackTrace(); 
}

The toURI() method will throw another exception known as URISyntaxException. You gotta handle it too.

If you run the above program you will get:

The protocol is: http
The file is: /path/data?key=value&key2=value2
The host name is: example.com
The port no. is: 123
The Query is: key=value&key2=value2
The Path is: /path/data
The User info is: username:password
The Reference is: fragid1
The Authority is: username:password@example.com:123
The URI format is: http://username:password@example.com:123/path/data?key=value&key2=value2#fragid1
The String representation is: http://username:password@example.com:123/path/data?key=value&key2=value2#fragid1

Hope that has covered all major get methods we needed to cover.

In order to understand what openConnection() does you need to first understand everything about Java URLConnection class.

Java URLConnection Class

URLConnection class is an abstract class. You make use of Java URLConnection class when you have a connection to establish between a URL and an application. You can read/write data to any resource that is pointed by a URL using Java URLConnection Class.

Creating a connection to a URL is a multistep process.

1. The first step is to create the connection object by using openConnection() on a URL.
2. Then you can manipulate the setup parameters and other general properties based on your requirement.
3. Then the actual connection is established using connect() method.
4. The remote object becomes accessible and now you can grab data from its content and header.

At times you might be required to grab on to the source code of a web page, that’s where getInputStream() method comes in handy. Using getInputStream() method of Java URLConnection Class you can grab all the data you need. Based upon your requirement then you can process stuff.

An example on how to display a source code of a web page using Java URLConnection class would help in making things more clearer.

Example to Grab Source Code from a Web Page

You need to catch two exceptions here namely: MalformedURLException and IOException

Here the goes the code to grab the source code:

try {

URL url = new URL("http://dumbitdude.com"); 
URLConnection urlc = url.openConnection(); 
BufferedReader br = new BufferedReader(new InputStreamReader(urlc.getInputStream())); 

          while(br.readLine()!=null){ 
               System.out.println(br.readLine()); 
           } 
} catch (MalformedURLException e) { 
e.printStackTrace();         
} catch (IOException io) { 
io.printStackTrace();
}

As you can see our first step was to use URL class’s openConnection() method. We grabbed a Java URLConnection instance from there that we used to grab the inputStream.

If you run the above program you will get its source code. I am not pasting my result here coz it was huge. You go ahead and try that.

Now that you have the source code you can use all sorts of things that you had learnt during our Regex and String chapter to match and sunder the things that you need.

HttpURLConnection Class

To start off with this I will tell you what HTTP is all about. It is a protocol that is an acronym for Hyper Text Transfer Protocol, and is used to facilitate communication between server and client.

The server could be a web server with an application to be downloaded and the client could be your browser. You send an HTTP request over to retrieve some data and the server, on the other hand, responds to your request by sending you data. It includes the status of the data too.

There are two famous HTTP request methods:

• GET
• POST

You might know that a GET method is used to retrieve data from a server. And that POST method is used when you are trying to store something on the server, or making sure your web server is accepting the things you send in a form or something.

We saw how Java URLConnection class helps you obtain a lot of info through a URL. If you want to make things a little more specific you can make use of HttpURLConnection class. The HttpURLConnection class extends Java URLConnection class directly. And it works only for “http” connections.

Using the HttpURLConnection class you can grab information related to any HTTP URL. You can grab its response code, response messages, status code or even header related info like header fields etc.

Since openConnection() method returns a URLConnection instance, we will simply use typecasting to use the method.

HttpURLConnection Example to Send GET and POST methods

Here’s the method to use when we are trying to send a GET method. If the server returns 200, that means that we have been able to make a connection successfully, that we can retrieve whatever data we want from the website now.

URL url = new URL("http://dumbitdude.com"); 
HttpURLConnection urlc = (HttpURLConnection) url.openConnection();

System.out.println("Sending a GET request.... ");
System.out.println(urlc.getResponseCode());

Notice how we have used typecasting here to return HttpURLConnection. Using its reference we have made a call to get a response code using getResponseCode() method.

NOTE: Do not forget to handle the exceptions found above.

If you run the above you will get:

Sending a GET request.... 
200

Here we were not required to use setRequestMethod(“GET”) explicitly because by default it is GET only.

You can follow it up by grabbing the source code or whatever info you wish to have.

In a similar fashion, you could send a POST method across as well.

URL url = new URL("http://dumbitdude.com/contact"); 
HttpURLConnection urlc = (HttpURLConnection) url.openConnection();

System.out.println("Sending a POST request...."); 
urlc.setRequestMethod("POST"); 
urlc.setDoOutput(true); 
DataOutputStream dos = new DataOutputStream(urlc.getOutputStream()); 
dos.writeUTF("Scottshak"); 
System.out.println(urlc.getResponseCode());

I have used a form page where the info will be delivered, and hence if you run the above program you will get:

Sending a POST request.... 
200

Example to Get Header Field Keys with Head Fields

If you wish to grab some header info the methods getHeaderFieldKey() and getHeaderField() are very useful. You just need to a run a “for loop” to display them all since they take integer as arguments.

URL url = new URL("http://dumbitdude.com"); 
HttpURLConnection urlc = (HttpURLConnection) url.openConnection(); 
for(int i = 0; i < 7; i++){ 
System.out.println(urlc.getHeaderFieldKey(i) + " = " + urlc.getHeaderField(i));
}

Do not forget to handle exceptions in the above. If we run the above program we will get:

Date = Thu, 06 Jul 2017 21:36:38 GMT
Server = Apache
X-Powered-By = PHP/5.4.45
Link = <http://dumbitdude.com/wp-json/>; rel="https://api.w.org/", <http://wp.me/8bUOO>; rel=shortlink
Transfer-Encoding = chunked
Content-Type = text/html; charset=UTF-8

which is the header information put against each key and field.

Java InetAddress Class

What if you want to grab the IP Address of a website? You could do that using the Java InetAddress Class. The Class represents IP Address.

There are some useful methods here like:

• getHostName()
• getHostAddress()
• getLocalHost()
• getByName()

Let’s see if we could grab the IP address of my website “http://dumbitdude.com” in an example:

InetAddress ia = InetAddress.getByName("dumbitdude.com"); 
System.out.println(ia.getHostName());
 System.out.println(ia.getHostAddress()); 
System.out.println(ia.getLocalHost());

If you run the above you will get the IP address, and the local IP Address too.

dumbitdude.com
199.79.62.63
DESKTOP-6ACK212/192.168.0.103

Cool right?

With that, I would like to conclude this chapter. The internet is your playground. Go experiment, child!