今日 Spring Boot 项目访问 Redis 报错了,错误信息为 “io.lettuce.core.RedisCommandExecutionException: ERR max number of clients reached”,详细错误堆栈信息如下:
2024/6/7 10:51:59 2024-06-07 10:51:59.211 service-user [lettuce-nioEventLoop-4-15] WARN io.lettuce.core.RedisClient - Cannot connect Redis Sentinel at redis://10.87.64.71:17001: java.util.concurrent.CompletionException: io.lettuce.core.RedisCommandExecutionException: ERR max number of clients reached 2024/6/7 10:51:59 2024-06-07 10:51:59.220 service-user [boundedElastic-2] WARN o.s.b.actuate.redis.RedisReactiveHealthIndicator - Redis health check failed 2024/6/7 10:51:59 org.springframework.data.redis.RedisConnectionFailureException: Unable to connect to Redis; nested exception is io.lettuce.core.RedisConnectionException: Cannot connect to a Redis Sentinel: [redis://10.87.64.30:17001, redis://10.87.64.65:17001, redis://10.87.64.71:17001] 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.translateException(LettuceConnectionFactory.java:1689) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1597) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getNativeConnection(LettuceConnectionFactory.java:1383) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$SharedConnection.getConnection(LettuceConnectionFactory.java:1366) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getSharedReactiveConnection(LettuceConnectionFactory.java:1117) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:509) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory.getReactiveConnection(LettuceConnectionFactory.java:103) 2024/6/7 10:51:59 at reactor.core.publisher.MonoSupplier.call(MonoSupplier.java:86) 2024/6/7 10:51:59 at reactor.core.publisher.FluxSubscribeOnCallable$CallableSubscribeOnSubscription.run(FluxSubscribeOnCallable.java:227) 2024/6/7 10:51:59 at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:68) 2024/6/7 10:51:59 at reactor.core.scheduler.SchedulerTask.call(SchedulerTask.java:28) 2024/6/7 10:51:59 at java.util.concurrent.FutureTask.run(FutureTask.java:266) 2024/6/7 10:51:59 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180) 2024/6/7 10:51:59 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) 2024/6/7 10:51:59 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 2024/6/7 10:51:59 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 2024/6/7 10:51:59 at java.lang.Thread.run(Thread.java:748) 2024/6/7 10:51:59 Caused by: io.lettuce.core.RedisConnectionException: Cannot connect to a Redis Sentinel: [redis://10.87.64.30:17001, redis://10.87.64.65:17001, redis://10.87.64.71:17001] 2024/6/7 10:51:59 at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:72) 2024/6/7 10:51:59 at io.lettuce.core.RedisConnectionException.create(RedisConnectionException.java:56) 2024/6/7 10:51:59 at io.lettuce.core.AbstractRedisClient.getConnection(AbstractRedisClient.java:330) 2024/6/7 10:51:59 at io.lettuce.core.RedisClient.connect(RedisClient.java:216) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.lambda$getConnection$1(StandaloneConnectionProvider.java:115) 2024/6/7 10:51:59 at java.util.Optional.orElseGet(Optional.java:267) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.StandaloneConnectionProvider.getConnection(StandaloneConnectionProvider.java:115) 2024/6/7 10:51:59 at org.springframework.data.redis.connection.lettuce.LettuceConnectionFactory$ExceptionTranslatingConnectionProvider.getConnection(LettuceConnectionFactory.java:1595) 2024/6/7 10:51:59 ... 15 common frames omitted 2024/6/7 10:51:59 Caused by: io.lettuce.core.RedisConnectionException: Cannot connect Redis Sentinel at redis://10.87.64.71:17001 2024/6/7 10:51:59 at io.lettuce.core.RedisClient.lambda$connectSentinelAsync$6(RedisClient.java:527) 2024/6/7 10:51:59 at reactor.core.publisher.Mono.lambda$onErrorMap$31(Mono.java:3733) 2024/6/7 10:51:59 at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:94) 2024/6/7 10:51:59 at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:106) 2024/6/7 10:51:59 at reactor.core.publisher.Operators.error(Operators.java:198) 2024/6/7 10:51:59 at reactor.core.publisher.MonoError.subscribe(MonoError.java:53) 2024/6/7 10:51:59 at reactor.core.publisher.Mono.subscribe(Mono.java:4400) 2024/6/7 10:51:59 at reactor.core.publisher.FluxOnErrorResume$ResumeSubscriber.onError(FluxOnErrorResume.java:103) 2024/6/7 10:51:59 at reactor.core.publisher.MonoCompletionStage.lambda$subscribe$0(MonoCompletionStage.java:77) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) 2024/6/7 10:51:59 at io.lettuce.core.AbstractRedisClient.lambda$null$5(AbstractRedisClient.java:458) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) 2024/6/7 10:51:59 at io.lettuce.core.protocol.RedisHandshakeHandler.lambda$fail$4(RedisHandshakeHandler.java:131) 2024/6/7 10:51:59 at io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:578) 2024/6/7 10:51:59 at io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:552) 2024/6/7 10:51:59 at io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:491) 2024/6/7 10:51:59 at io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:184) 2024/6/7 10:51:59 at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:95) 2024/6/7 10:51:59 at io.netty.channel.DefaultChannelPromise.addListener(DefaultChannelPromise.java:30) 2024/6/7 10:51:59 at io.lettuce.core.protocol.RedisHandshakeHandler.fail(RedisHandshakeHandler.java:130) 2024/6/7 10:51:59 at io.lettuce.core.protocol.RedisHandshakeHandler.lambda$channelActive$3(RedisHandshakeHandler.java:100) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) 2024/6/7 10:51:59 at io.lettuce.core.RedisHandshake.lambda$tryHandshakeResp3$1(RedisHandshake.java:105) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:760) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:736) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:474) 2024/6/7 10:51:59 at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1977) 2024/6/7 10:51:59 at io.lettuce.core.protocol.AsyncCommand.doCompleteExceptionally(AsyncCommand.java:139) 2024/6/7 10:51:59 at io.lettuce.core.protocol.AsyncCommand.completeResult(AsyncCommand.java:120) 2024/6/7 10:51:59 at io.lettuce.core.protocol.AsyncCommand.complete(AsyncCommand.java:111) 2024/6/7 10:51:59 at io.lettuce.core.protocol.CommandWrapper.complete(CommandWrapper.java:63) 2024/6/7 10:51:59 at io.lettuce.core.protocol.CommandHandler.complete(CommandHandler.java:746) 2024/6/7 10:51:59 at io.lettuce.core.protocol.CommandHandler.decode(CommandHandler.java:681) 2024/6/7 10:51:59 at io.lettuce.core.protocol.CommandHandler.channelRead(CommandHandler.java:598) 2024/6/7 10:51:59 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 2024/6/7 10:51:59 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 2024/6/7 10:51:59 at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) 2024/6/7 10:51:59 at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) 2024/6/7 10:51:59 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) 2024/6/7 10:51:59 at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) 2024/6/7 10:51:59 at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) 2024/6/7 10:51:59 at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:166) 2024/6/7 10:51:59 at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:722) 2024/6/7 10:51:59 at io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:658) 2024/6/7 10:51:59 at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:584) 2024/6/7 10:51:59 at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:496) 2024/6/7 10:51:59 at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:986) 2024/6/7 10:51:59 at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) 2024/6/7 10:51:59 at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) 2024/6/7 10:51:59 ... 1 common frames omitted 2024/6/7 10:51:59 Caused by: io.lettuce.core.RedisCommandExecutionException: ERR max number of clients reached 2024/6/7 10:51:59 at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:147) 2024/6/7 10:51:59 at io.lettuce.core.internal.ExceptionFactory.createExecutionException(ExceptionFactory.java:116) 2024/6/7 10:51:59 ... 22 common frames omitted
上述的“io.lettuce.core.RedisCommandExecutionException: ERR max number of clients reached”异常表示你的 Redis 服务器已经达到了它所能处理的最大客户端连接数。每个 Redis 服务器实例都有一个配置好的最大客户端连接数限制,这个限制是为了防止资源耗尽,比如内存或文件描述符。
问题分析如下:
配置限制:Redis 服务器有一个 maxclients 配置项,它定义了服务器允许的最大并发连接数。当达到这个限制时,任何新的连接尝试都会失败,并返回上述错误。
连接泄露:如果你的应用程序没有正确关闭不再需要的 Redis 连接,那么这些连接可能会保持打开状态,最终导致达到最大连接数。
并发连接过多:如果你的应用程序或者多个应用程序实例同时尝试建立大量的连接,而这些连接的数量超过了 Redis 服务器设置的 maxclients 值,就会出现这个错误。
Redis 服务器资源限制:在某些情况下,操作系统级别的资源限制(比如文件描述符限制)也可能间接导致这个问题。
为了解决这个问题,你可以采取以下措施:
增加 maxclients 值:你可以通过修改 Redis 配置文件中的 maxclients 配置项来增加最大连接数,但这通常需要谨慎考虑,因为太多的连接会消耗更多的资源。
检查并修复连接泄露:确保你的应用程序在不再需要 Redis 连接时正确关闭它们,使用连接池可以更有效地管理连接。通常,Redis 连接是由对应的库来管理,不需要我们手动管理,如:RedisTemplate。
优化应用程序的并发连接数:如果你的应用程序尝试建立大量的并发连接,考虑是否可以减少并发连接数,或者通过连接复用等方式优化。
检查操作系统资源限制:确保你的操作系统没有设置过低的文件描述符限制或其他相关资源限制。
监控和警报:设置监控和警报,以便在接近最大连接数时得到通知,从而可以及时调整配置或优化应用程序。