Accident description
SMS service is extracted separately to provide HTTP interface access. Load balancing is provided through haproxy
Occasionally in the log:
java.net.SocketException: Unexpected end of file from server
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:792)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.http.HttpClient.parseHTTPHeader(HttpClient.java:789)
at sun.net.www.http.HttpClient.parseHTTP(HttpClient.java:647)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1569)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1474)
At that time did not care too much, until the request SMS service is not available, to investigate the problem
Because the HTTP request code did not set the timeout, haproxy hung up the request and the whole service was unavailable
Exception code
try {
u = new URL(url);
con = (HttpURLConnection) u.openConnection();
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setDoInput(true);
con.setUseCaches(false);
con.setRequestProperty("Content-Type", "application/json");
OutputStreamWriter osw = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
osw.write(msg);
osw.flush();
osw.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (con != null) {
con.disconnect();
}
}
Restoration plan
**Add timeouts. All IO operations should add timeouts**
try {
u = new URL(url);
con = (HttpURLConnection) u.openConnection();
con.setRequestMethod("POST");
con.setDoOutput(true);
con.setDoInput(true);
con.setUseCaches(false);
con.setConnectTimeout(3000);
con.setReadTimeout(3000);
con.setRequestProperty("Content-Type", "application/json");
OutputStreamWriter osw = new OutputStreamWriter(con.getOutputStream(), "UTF-8");
osw.write(msg);
osw.flush();
osw.close();
} catch (Exception e) {
e.printStackTrace();
} finally {
if (con != null) {
con.disconnect();
}
}
Second, because the old code directly used the net package, I don’t think it’s very easy to use, so I directly upgraded to httpclient fluent
try {
return Request.Post(url)
.useExpectContinue()
.version(HttpVersion.HTTP_1_1)
.bodyString(param, ContentType.APPLICATION_JSON)
.connectTimeout(3000)
.socketTimeout(3000)
.execute()
.returnContent()
.asString();
} catch (IOException e) {
LOGGER.error("request address:{},parameters:{}",url,param);
LOGGER.error("Error in post request",e);
}
POM dependency
<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>fluent-hc</artifactId>
<version>4.5.3</version>
</dependency>