异常处理

Hystrix异常类型

  • HystrixRuntimeException
  • HystrixBadRequestException
  • HystrixTimeoutException
  • RejectedExecutionException

HystrixRuntimeException

HystrixCommand失败时抛出, 不会触发fallback.

HystrixBadRequestException

用提供的参数或状态表示错误的异常, 而不是执行失败. 与其他HystrixCommand抛出的异常不同, 这个异常不会触发fallback, 也不会记录进failure的指标, 因而也不会触发断路器,

应该在用户输入引起的错误是抛出, 否则会它与容错和后退行为的目的相悖.

不会触发fallback, 也不会记录到错误的指标中, 也不会触发断路器.

RejectedExecutionException

线程池发生reject时抛出

HystrixTimeoutException

HystrixCommand.run()或者HystrixObservableCommand.construct()时抛出, 会记录timeout的次数. 如果希望某些类型的失败被记录为timeout, 应该将这些类型的失败包装为HystrixTimeoutException

异常处理

ignoreExceptions

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
final Func1<Throwable, Observable<R>> handleFallback = new Func1<Throwable, Observable<R>>() {
@Override
public Observable<R> call(Throwable t) {
circuitBreaker.markNonSuccess();
Exception e = getExceptionFromThrowable(t);
executionResult = executionResult.setExecutionException(e);
if (e instanceof RejectedExecutionException) {
return handleThreadPoolRejectionViaFallback(e);
} else if (t instanceof HystrixTimeoutException) {
return handleTimeoutViaFallback();
} else if (t instanceof HystrixBadRequestException) {
return handleBadRequestByEmittingError(e);
} else {
/*
* Treat HystrixBadRequestException from ExecutionHook like a plain HystrixBadRequestException.
*/
if (e instanceof HystrixBadRequestException) {
eventNotifier.markEvent(HystrixEventType.BAD_REQUEST, commandKey);
return Observable.error(e);
}
return handleFailureViaFallback(e);
}
}
};

Feign中响应状态码处理

Feign使用SynchronousMethodHandler做请求的执行和响应的处理. 响应处理的部分, 对[200, 300)区间的状态, 会将response返回; 如果是404, 根据@FeignClientdecode404(默认为false)和方法返回值判断是否熔断, 如果响应返回404, decodefalse, 同时方法返回值不是void, 会包装成FeignException抛出; 其他的状态, 通过包装成FeignException抛出.

FeignExceptionRuntimeException的实现, 如果没有ignore的话, 会计入熔断器的计算中.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
final class SynchronousMethodHandler implements MethodHandler {
Object executeAndDecode(RequestTemplate template) throws Throwable {
...
if (response.status() >= 200 && response.status() < 300) {
if (void.class == metadata.returnType()) {
return null;
} else {
return decode(response);
}
} else if (decode404 && response.status() == 404 && void.class != metadata.returnType()) {
return decode(response);
} else {
throw errorDecoder.decode(metadata.configKey(), response);
}
...
}
}

Ribbon中响应状态码处理

在Zuul中, 路由使用Ribbon做负载均衡, 同时使用Hystrix做断路器, 使用RibbonCommand接口的实现. RibbonCommand的实现并没有对响应编码封装异常, 因此也不会触发熔断器.

AbstractRibbonCommandRibbonCommand的抽象实现, 所有其他实现的父类. 核心run()方法并没有针对响应编码重新封装异常.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
public abstract class AbstractRibbonCommand<LBC extends AbstractLoadBalancerAwareClient<RQ, RS>, RQ extends ClientRequest, RS extends HttpResponse>
extends HystrixCommand<ClientHttpResponse> implements RibbonCommand {
...
@Override
protected ClientHttpResponse run() throws Exception {
final RequestContext context = RequestContext.getCurrentContext();

RQ request = createRequest();
RS response;

boolean retryableClient = this.client instanceof AbstractLoadBalancingClient
&& ((AbstractLoadBalancingClient)this.client).isClientRetryable((ContextAwareRequest)request);

if (retryableClient) {
response = this.client.execute(request, config);
} else {
response = this.client.executeWithLoadBalancer(request, config);
}
context.set("ribbonResponse", response);

// Explicitly close the HttpResponse if the Hystrix command timed out to
// release the underlying HTTP connection held by the response.
//
if (this.isResponseTimedOut()) {
if (response != null) {
response.close();
}
}

return new RibbonHttpResponse(response);
}
...
}

Observable.error(ex)会捕获run()方法抛出的异常.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
public abstract class HystrixCommand<R> extends AbstractCommand<R> implements HystrixExecutable<R>, HystrixInvokableInfo<R>, HystrixObservable<R> {
...
final protected Observable<R> getExecutionObservable() {
return Observable.defer(new Func0<Observable<R>>() {
@Override
public Observable<R> call() {
try {
return Observable.just(run());
} catch (Throwable ex) {
return Observable.error(ex);
}
}
}).doOnSubscribe(new Action0() {
@Override
public void call() {
// Save thread on which we get subscribed so that we can interrupt it later if needed
executionThread.set(Thread.currentThread());
}
});
}
...
}

Hystrix 超时处理

在Hystrix版本1.4之前, Seamphore策略是不支持超时的. 目前spring-cloud-netflix的1.4.4中使用的是1.5.12

如果开启了timeout, HystrixCommand会lift一个HystrixObservableTimeoutOperatorObservable中.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
abstract class AbstractCommand<R> implements HystrixInvokableInfo<R>, HystrixObservable<R> {
private Observable<R> executeCommandAndObserve(final AbstractCommand<R> _cmd) {
...
Observable<R> execution;
if (properties.executionTimeoutEnabled().get()) {
execution = executeCommandWithSpecifiedIsolation(_cmd)
.lift(new HystrixObservableTimeoutOperator<R>(_cmd));
} else {
execution = executeCommandWithSpecifiedIsolation(_cmd);
}

return execution.doOnNext(markEmits)
.doOnCompleted(markOnCompleted)
.onErrorResumeNext(handleFallback)
.doOnEach(setRequestContext);
}
}

这个HystrixObservableTimeoutOperator会添加注册TimeListener. TimeListener是以tick的方式运行, 即启动一个线程延迟executionTimeoutInMilliseconds运行, 然后每次在executionTimeoutInMilliseconds + n * executionTimeoutInMilliseconds时运行.

如果判断操作超时? 看tick方法的实现, 线程每次运行时, 尝试修改Command的状态从NOT_EXECUTEDTIMED_OUT. 如果成功, 说明运行超时. 最后抛出HystrixTimeoutException异常, 被handleFallback处理.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
// if we can go from NOT_EXECUTED to TIMED_OUT then we do the timeout codepath
// otherwise it means we lost a race and the run() execution completed or did not start
if (originalCommand.isCommandTimedOut.compareAndSet(TimedOutStatus.NOT_EXECUTED, TimedOutStatus.TIMED_OUT)) {
// report timeout failure
originalCommand.eventNotifier.markEvent(HystrixEventType.TIMEOUT, originalCommand.commandKey);
// shut down the original request
s.unsubscribe();
final HystrixContextRunnable timeoutRunnable = new HystrixContextRunnable(originalCommand.concurrencyStrategy, hystrixRequestContext, new Runnable() {
@Override
public void run() {
child.onError(new HystrixTimeoutException());
}
});
timeoutRunnable.run();
//if it did not start, then we need to mark a command start for concurrency metrics, and then issue the timeout
}