OkHttp源码学习随笔

OkHttp是什么?

简介

OkHttp是一款优秀的HTTP框架,它支持get请求和post请求,支持基于Http的文件上传和下载,支持加载图片,支持下载文件透明的GZIP压缩,支持响应缓存避免重复的网络请求,支持使用连接池来降低响应延迟问题。OkHttp由Square公司开发,是目前Android最热门的网络框架之一。

官网网址:OKHttp官网

Github地址:Github

特点

  1. 支持HTTP2/SPDY
  2. socket自动选择最好路线,并支持自动重连
  3. 拥有自动维护的socket连接池,减少握手次数
  4. 拥有队列线程池,轻松写并发
  5. 拥有Interceptors轻松处理请求与响应(比如透明GZIP压缩)基于Headers的缓存策略

OkHttp怎么用?

1、gradle引入库,implementation ‘com.squareup.okhttp3:okhttp:3.11.0’

2、初始化OkHttpClient对象

1
2
3
4
5
client = new OkHttpClient.Builder()
.connectTimeout(15, TimeUnit.SECONDS)
.readTimeout(15, TimeUnit.SECONDS)
.writeTimeout(15, TimeUnit.SECONDS)
.build();

同步请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
public void okHttpSync() {
Request request = new Request.Builder()
.url("https://www.baidu.com")
.build();
Call call = client.newCall(request);
try {
Response response = call.execute();
if (response.isSuccessful()) {
System.out.println("response.code()==" + response.code());
System.out.println("response.heard()==" + response.headers());
System.out.println("response.message()==" + response.message());
System.out.println("res==" + response.body().string());
}
} catch (IOException e) {
e.printStackTrace();
}
}

异步请求

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
public void okHttpAsync() {
Request request = new Request.Builder()
.url("https://www.baidu.com")
.build();
Call call = client.newCall(request);
call.enqueue(new Callback() {
@Override
public void onFailure(Call call, IOException e) {
needCancelled.set(true);
System.out.println("url==" + call.request().url());
}

@Override
public void onResponse(Call call, Response response) throws IOException {
if (response.isSuccessful()) {
System.out.println("response.code()==" + response.code());
System.out.println("response.heard()==" + response.headers());
System.out.println("response.message()==" + response.message());
System.out.println("res==" + response.body().string());
needCancelled.set(true);
}
}
});
}

详细的OkHttp使用可参考OKHttp使用详解

OkHttp核心执行流程是怎样?

关键类功能说明

功能说明
OKHttpClient 里面包含了很多对象,OKhttp的很多功能模块都包装进这个类,让这个类单独提供对外的API,使用Builder模型构建
Request、Response 抽象的网络输入及响应模型
Call HTTP请求任务封装,是一个接口
RealCall Call的实现,实现execute()同步方法、enqueue(Callback responseCallback)异步方法, getResponseWithInterceptorChain() 获取拦截器响应
AsyncCall RealCall的内部类。继承了Runnable接口,后续在异步的线程池中执行
Dispatcher 核心调度类,内部维护为了readyAsyncCalls、runningAsyncCalls、runningSyncCalls队列,实际RealCall后续也是调用该类进行同步、异步的具体实现。内部维护了一个线程池,限制了最大并发数maxRequests=64。
RealInterceptorChain 拦截器链,维护了一个interceptors队列,每次proceed通过index + 1会执行下一拦截器的intercept方法
RetryAndFollowUpInterceptor 负责失败重连以及重定向
BridgeInterceptor 负责对Request和Response报文进行加工
CacheInterceptor 负责缓存拦截器
ConnectInterceptor 负责维护连接拦截器
CallServerInterceptor 负责最后网络IO的读写

代码执行流程

image

1、通过Builder模式统一构建OkHttpClient对象

2、通过Call,实现类RealCall进行请求发送

3、RealCall通过调用了Dispatcher的execute()及enqueue()方法进行同步及异步的请求

4、最终调用ReallCall的getResponseWithInterceptorChain()方法进行拦截链的拦截

5、依次通过重定向拦截器、桥接拦截器、缓存拦截器、连接拦截器、网络拦截器依次进行处理

6、最后通过intercept的return往回返回Response,最终返回给客户端请求的结果

OkHttp如何进行线程调度控制?

线程调度

在Dispatcher中维护了一个线程池,异步的请求会将任务加入到线程池中。

1
2
3
4
5
6
7
public synchronized ExecutorService executorService() {
if (executorService == null) {
executorService = new ThreadPoolExecutor(0, Integer.MAX_VALUE, 60, TimeUnit.SECONDS,
new SynchronousQueue<Runnable>(), Util.threadFactory("OkHttp Dispatcher", false));
}
return executorService;
}

默认的最大并发数为maxRequests=64,如果超过限制会加入到等待队列中,执行异步的方法如下

1
2
3
4
5
6
7
8
synchronized void enqueue(AsyncCall call) {
if (runningAsyncCalls.size() < maxRequests && runningCallsForHost(call) < maxRequestsPerHost) {
runningAsyncCalls.add(call);
executorService().execute(call);
} else {
readyAsyncCalls.add(call);
}
}

最后线程池执行AsyncCall中的execute()方法,如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
@Override protected void execute() {
boolean signalledCallback = false;
try {
Response response = getResponseWithInterceptorChain();
if (retryAndFollowUpInterceptor.isCanceled()) {
signalledCallback = true;
responseCallback.onFailure(RealCall.this, new IOException("Canceled"));
} else {
signalledCallback = true;
responseCallback.onResponse(RealCall.this, response);
}
} catch (IOException e) {
if (signalledCallback) {
// Do not signal the callback twice!
Platform.get().log(INFO, "Callback failure for " + toLoggableString(), e);
} else {
eventListener.callFailed(RealCall.this, e);
responseCallback.onFailure(RealCall.this, e);
}
} finally {
client.dispatcher().finished(this);
}
}

队列机制

Dispathcer中维护了3个队列,分别为异步等待队列、异步执行队列、同步执行队列。

1
2
3
4
5
6
7
8
/** Ready async calls in the order they'll be run. */
private final Deque<AsyncCall> readyAsyncCalls = new ArrayDeque<>();

/** Running asynchronous calls. Includes canceled calls that haven't finished yet. */
private final Deque<AsyncCall> runningAsyncCalls = new ArrayDeque<>();

/** Running synchronous calls. Includes canceled calls that haven't finished yet. */
private final Deque<RealCall> runningSyncCalls = new ArrayDeque<>();

不管是同步还是异步,最终在finally块都会调用dispatcher的finished方法,会移除掉该队列任务,最后实现如下

1
2
3
4
5
6
7
8
9
10
11
12
int runningCallsCount;
Runnable idleCallback;
synchronized (this) {
if (!calls.remove(call)) throw new AssertionError("Call wasn't in-flight!");
if (promoteCalls) promoteCalls();
runningCallsCount = runningCallsCount();
idleCallback = this.idleCallback;
}

if (runningCallsCount == 0 && idleCallback != null) {
idleCallback.run();
}

在finish中会再调用promoteCalls方法,会重新检索准备中的队列,将队列加入到线程中

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
private void promoteCalls() {
if (runningAsyncCalls.size() >= maxRequests) return; // Already running max capacity.
if (readyAsyncCalls.isEmpty()) return; // No ready calls to promote.

for (Iterator<AsyncCall> i = readyAsyncCalls.iterator(); i.hasNext(); ) {
AsyncCall call = i.next();

if (runningCallsForHost(call) < maxRequestsPerHost) {
i.remove();
runningAsyncCalls.add(call);
executorService().execute(call);
}

if (runningAsyncCalls.size() >= maxRequests) return; // Reached max capacity.
}
}

OkHttp的拦截器及调用链是怎么执行?

调用链执行流程

通过上述的分析,我们知道不管同步还是异步,最终调用到的都是RealCall的getResponseWithInterceptorChain()方法,如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
Response getResponseWithInterceptorChain() throws IOException {
// Build a full stack of interceptors.
List<Interceptor> interceptors = new ArrayList<>();
interceptors.addAll(client.interceptors());
interceptors.add(retryAndFollowUpInterceptor);
interceptors.add(new BridgeInterceptor(client.cookieJar()));
interceptors.add(new CacheInterceptor(client.internalCache()));
interceptors.add(new ConnectInterceptor(client));
if (!forWebSocket) {
interceptors.addAll(client.networkInterceptors());
}
interceptors.add(new CallServerInterceptor(forWebSocket));

Interceptor.Chain chain = new RealInterceptorChain(interceptors, null, null, null, 0,
originalRequest, this, eventListener, client.connectTimeoutMillis(),
client.readTimeoutMillis(), client.writeTimeoutMillis());

return chain.proceed(originalRequest);
}

其中定义了拦截器集合及RealInterceptorChain拦截链,具体执行了拦截链的proceed方法,如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
public Response proceed(Request request, StreamAllocation streamAllocation, HttpCodec httpCodec,
RealConnection connection) throws IOException {
if (index >= interceptors.size()) throw new AssertionError();

calls++;

// If we already have a stream, confirm that the incoming request will use it.
if (this.httpCodec != null && !this.connection.supportsUrl(request.url())) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must retain the same host and port");
}

// If we already have a stream, confirm that this is the only call to chain.proceed().
if (this.httpCodec != null && calls > 1) {
throw new IllegalStateException("network interceptor " + interceptors.get(index - 1)
+ " must call proceed() exactly once");
}

// Call the next interceptor in the chain.
RealInterceptorChain next = new RealInterceptorChain(interceptors, streamAllocation, httpCodec,
connection, index + 1, request, call, eventListener, connectTimeout, readTimeout,
writeTimeout);
Interceptor interceptor = interceptors.get(index);
Response response = interceptor.intercept(next);

// Confirm that the next interceptor made its required call to chain.proceed().
if (httpCodec != null && index + 1 < interceptors.size() && next.calls != 1) {
throw new IllegalStateException("network interceptor " + interceptor
+ " must call proceed() exactly once");
}

// Confirm that the intercepted response isn't null.
if (response == null) {
throw new NullPointerException("interceptor " + interceptor + " returned null");
}

if (response.body() == null) {
throw new IllegalStateException(
"interceptor " + interceptor + " returned a response with no body");
}

return response;
}

1、先判断是否超过list的size,如果超过则遍历结束,如果没有超过则继续执行

2、calls+1

3、new了一个RealInterceptorChain,其中然后下标index+1

4、从list取出下一个interceptor对象

5、执行interceptor的intercept方法

总结一下就是每一个RealInterceptorChain对应一个interceptor,然后每一个interceptor再产生下一个RealInterceptorChain,直到List迭代完成。

拦截器

image

从上面的调用关系可以看出除了红色圈出的拦截器之外都是系统提供的拦截器,这整个过程是递归的执行过程,在 CallServerInterceptor 中得到最终的 Response 之后,将 response 按递归逐级进行返回,期间会经过 NetworkInterceptor 最后到达 Application Interceptor 。

OkHttp是如何进行数据缓存?

缓存策略

OkHttp使用了CacheInterceptor拦截器进行数据缓存的控制使用了CacheStrategy实现了上面的流程图,它根据之前缓存的结果与当前将要发送Request的header进行策略,并得出是否进行请求的结果。根据输出的networkRequest和cacheResponse的值是否为null给出不同的策略,如下:

networkRequest cacheResponse result 结果
null null only-if-cached (表明不进行网络请求,且缓存不存在或者过期,一定会返回503错误)
null non-null 不进行网络请求,直接返回缓存,不请求网络
non-null null 需要进行网络请求,而且缓存不存在或者过去,直接访问网络
non-null non-null Header中包含ETag/Last-Modified标签,需要在满足条件下请求,还是需要访问网络

缓存算法

通过分析CacheInterceptor拦截器的intercept方法,我们可以发现具体的缓存都是使用了Cache类进行,最后具体的实现在DiskLruCache类中。缓存实际上是一个比较复杂的逻辑,单独的功能块,实际上不属于OKhttp上的功能,实际上是通过是http协议和DiskLruCache做了处理。LinkedHashMap可以实现LRU算法,并且在这个case里,它被用作对DiskCache的内存索引

有兴趣可以参考如下2篇文章的具体实现:

OKHttp源码解析(六)–中阶之缓存基础

OKHttp源码解析(七)–中阶之缓存机制

OkHttp的连接池复用机制是怎么样?

链路

RealConnection是Connection的实现类,代表着链接socket的链路,如果拥有了一个RealConnection就代表了我们已经跟服务器有了一条通信链路,而且通过
RealConnection代表是连接socket链路,RealConnection对象意味着我们已经跟服务端有了一条通信链路。
另外StreamAllocation类为流的桥梁,在RetryAndFollowUpInterceptor中进行初始化,在ConnectInterceptor中进行newStream操作,具体的连接拦截器代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
public Response intercept(Chain chain) throws IOException {
RealInterceptorChain realChain = (RealInterceptorChain) chain;
Request request = realChain.request();
StreamAllocation streamAllocation = realChain.streamAllocation();

// We need the network to satisfy this request. Possibly for validating a conditional GET.
boolean doExtensiveHealthChecks = !request.method().equals("GET");
HttpCodec httpCodec = streamAllocation.newStream(client, chain, doExtensiveHealthChecks);
RealConnection connection = streamAllocation.connection();

return realChain.proceed(request, streamAllocation, httpCodec, connection);
}

newStream创建留最后会调用到findConnection方法,这里面是连接复用的关键,如果再连接池中找到能复用的连接,则直接返回。
否则将RealConnection加入到链接池ConnectionPool中,具体代码如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
private RealConnection findConnection(int connectTimeout, int readTimeout, int writeTimeout,
int pingIntervalMillis, boolean connectionRetryEnabled) throws IOException {
boolean foundPooledConnection = false;
RealConnection result = null;
Route selectedRoute = null;
Connection releasedConnection;
Socket toClose;
synchronized (connectionPool) {
if (released) throw new IllegalStateException("released");
if (codec != null) throw new IllegalStateException("codec != null");
if (canceled) throw new IOException("Canceled");

// Attempt to use an already-allocated connection. We need to be careful here because our
// already-allocated connection may have been restricted from creating new streams.
releasedConnection = this.connection;
toClose = releaseIfNoNewStreams();
if (this.connection != null) {
// We had an already-allocated connection and it's good.
result = this.connection;
releasedConnection = null;
}
if (!reportedAcquired) {
// If the connection was never reported acquired, don't report it as released!
releasedConnection = null;
}

if (result == null) {
// Attempt to get a connection from the pool.
Internal.instance.get(connectionPool, address, this, null);
if (connection != null) {
foundPooledConnection = true;
result = connection;
} else {
selectedRoute = route;
}
}
}
closeQuietly(toClose);

if (releasedConnection != null) {
eventListener.connectionReleased(call, releasedConnection);
}
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
}
if (result != null) {
// If we found an already-allocated or pooled connection, we're done.
return result;
}

// If we need a route selection, make one. This is a blocking operation.
boolean newRouteSelection = false;
if (selectedRoute == null && (routeSelection == null || !routeSelection.hasNext())) {
newRouteSelection = true;
routeSelection = routeSelector.next();
}

synchronized (connectionPool) {
if (canceled) throw new IOException("Canceled");

if (newRouteSelection) {
// Now that we have a set of IP addresses, make another attempt at getting a connection from
// the pool. This could match due to connection coalescing.
List<Route> routes = routeSelection.getAll();
for (int i = 0, size = routes.size(); i < size; i++) {
Route route = routes.get(i);
Internal.instance.get(connectionPool, address, this, route);
if (connection != null) {
foundPooledConnection = true;
result = connection;
this.route = route;
break;
}
}
}

if (!foundPooledConnection) {
if (selectedRoute == null) {
selectedRoute = routeSelection.next();
}

// Create a connection and assign it to this allocation immediately. This makes it possible
// for an asynchronous cancel() to interrupt the handshake we're about to do.
route = selectedRoute;
refusedStreamCount = 0;
result = new RealConnection(connectionPool, selectedRoute);
acquire(result, false);
}
}

// If we found a pooled connection on the 2nd time around, we're done.
if (foundPooledConnection) {
eventListener.connectionAcquired(call, result);
return result;
}

// Do TCP + TLS handshakes. This is a blocking operation.
result.connect(connectTimeout, readTimeout, writeTimeout, pingIntervalMillis,
connectionRetryEnabled, call, eventListener);
routeDatabase().connected(result.route());

Socket socket = null;
synchronized (connectionPool) {
reportedAcquired = true;

// Pool the connection.
Internal.instance.put(connectionPool, result);

// If another multiplexed connection to the same address was created concurrently, then
// release this connection and acquire that one.
if (result.isMultiplexed()) {
socket = Internal.instance.deduplicate(connectionPool, address, this);
result = connection;
}
}
closeQuietly(socket);

eventListener.connectionAcquired(call, result);
return result;
}

连接池

OkHttp中使用ConnectionPool管理http和http/2的链接,以便减少网络请求延迟。同一个address将共享同一个connection。该类实现了复用连接的目标。一个OkHttpClient只包含一个ConnectionPool,其实例化也是在OkHttpClient的过程。这里说一下ConnectionPool各个方法的调用并没有直接对外暴露,而是通过OkHttpClient的Internal接口统一对外暴露。

1、获取连接使用get方法,或获取是否有合适的链接,否则返回null,具体实现如下:

1
2
3
4
5
6
7
8
9
10
RealConnection get(Address address, StreamAllocation streamAllocation, Route route) {
assert (Thread.holdsLock(this));
for (RealConnection connection : connections) {
if (connection.isEligible(address, route)) {
streamAllocation.acquire(connection, true);
return connection;
}
}
return null;
}

2、加入连接使用put方法,并且会是会触发cleanupRunnable,清理连接。具体实现如下:

1
2
3
4
5
6
7
8
void put(RealConnection connection) {
assert (Thread.holdsLock(this));
if (!cleanupRunning) {
cleanupRunning = true;
executor.execute(cleanupRunnable);
}
connections.add(connection);
}

3、具体的连接回收机制,首先统计空闲连接数量,然后通过for循环查找最长空闲时间的连接以及对应空闲时长,然后判断是否超出最大空闲连接数(maxIdleConnections)或者或者超过最大空闲时间(keepAliveDurationNs),满足其一则清除最长空闲时长的连接。如果不满足清理条件,则返回一个对应等待时间。具体的实现如下:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
long cleanup(long now) {
int inUseConnectionCount = 0;
int idleConnectionCount = 0;
RealConnection longestIdleConnection = null;
long longestIdleDurationNs = Long.MIN_VALUE;

// Find either a connection to evict, or the time that the next eviction is due.
synchronized (this) {
for (Iterator<RealConnection> i = connections.iterator(); i.hasNext(); ) {
RealConnection connection = i.next();

// If the connection is in use, keep searching.
if (pruneAndGetAllocationCount(connection, now) > 0) {
inUseConnectionCount++;
continue;
}

idleConnectionCount++;

// If the connection is ready to be evicted, we're done.
long idleDurationNs = now - connection.idleAtNanos;
if (idleDurationNs > longestIdleDurationNs) {
longestIdleDurationNs = idleDurationNs;
longestIdleConnection = connection;
}
}

if (longestIdleDurationNs >= this.keepAliveDurationNs
|| idleConnectionCount > this.maxIdleConnections) {
// We've found a connection to evict. Remove it from the list, then close it below (outside
// of the synchronized block).
connections.remove(longestIdleConnection);
} else if (idleConnectionCount > 0) {
// A connection will be ready to evict soon.
return keepAliveDurationNs - longestIdleDurationNs;
} else if (inUseConnectionCount > 0) {
// All connections are in use. It'll be at least the keep alive duration 'til we run again.
return keepAliveDurationNs;
} else {
// No connections, idle or in use.
cleanupRunning = false;
return -1;
}
}

closeQuietly(longestIdleConnection.socket());

// Cleanup again immediately.
return 0;
}

OkHttp的底层网络实现是什么?

1、OkHttp使用okio进行io的操作。okio是由square公司开发的,它补充了java.io和java.nio的不足,以便能够更加方便,快速的访问、存储和处理你的数据。OKHttp底层也是用该库作为支持。而且okio使用起来很简单,减少了很多io操作的基本代码,并且对内存和CPU使用做了优化。

2、没有依赖其他的关于Http实现的库,底层使用了Socket,自己实现了Http1.X及2.X的协议。

OkHttp中代码运用了那些设计模式,有什么巧妙的设计?

1、建造者模式

不管是OkHttpClient对象的创建还是Request对象、Respone对象,都使用了建造者模式,将复杂的对象创建统一在不同方法中,使得创建的过程更加简单。

2、外观模式
OkHttpClient对外提供了统一的调度,屏蔽了内部的实现,使得使用该网络库简单便捷。

3、责任链模式
OkHttp中的拦截器使用了责任链模式,将不同的拦截器独立实现,动态组成链的调用形式。责任清晰,可动态扩展。

为什么要用OkHttp?

目前Android开发中,主要的网络框架有HttpClient、Volley、HttpURLConnection、OkHttp。

其中Android早就不推荐httpclient,5.0之后干脆废弃,6.0删除了HttpClient。所以HttpClient不考虑。Volley框架现在也已经不再升级了,故目前考虑使用的有、HttpURLConnection及OkHttp。

相对HttpURLConnection,OkHttp使用更加便捷及灵活,且第三方社区活跃,相关资料齐全,成熟稳定性高。OkHttp也得到了官方的认可,并在不断优化更新,所以建议应用优先选择OkHttp作为网络框架。

总结

思考

在项目的开发过程中,我们经常使用到大量的第三方框架,但可能知其然不知其所以然,通过不断的思考反问为什么,从而去研究源码中的实现。能让我们对框架更加运用自如及解决一些底层疑难的问题。

参考资料

OKHttp使用详解

OKHTTP结合官网示例分析两种自定义拦截器的区别

OKHttp源码解析(一)系列