前言
本文将会使用 SpringCloud Gateway 网关组件配合 Nacos 实现灰度发布(金丝雀发布)
环境搭建
创建子模块服务提供者 provider,网关模块 gateway
父项目
pom.xml 配置
4.0.0 com.example spring-gateway-demo 0.0.1-SNAPSHOT pom spring-gateway-demo spring-gateway-demo 11 11 11 3.8.1 2.3.7.RELEASE 2.2.2.RELEASE Hoxton.SR9 2.2.0.RELEASE provider gateway com.alibaba.cloud spring-cloud-starter-alibaba-nacos-config com.alibaba.cloud spring-cloud-starter-alibaba-nacos-discovery org.springframework.cloud spring-cloud-dependencies ${spring-cloud.version} pom import org.springframework.boot spring-boot-dependencies ${spring-boot.version} pom import com.alibaba.cloud spring-cloud-alibaba-dependencies ${spring-cloud-alibaba.version} pom import
服务提供者 provider
这里我们计划引入 nacos, 所以先创建一个 nacos 配置文件 dataId 为 provider.properties, 这里用默认的命名空间 public, 默认分组 DEFAULT_GROUP
version=2
provider 的 pom 配置依赖
org.springframework.boot spring-boot-starter-web org.springframework.boot spring-boot-starter-test test org.junit.vintage junit-vintage-engine
application.yml
server: port: 9001 spring: application: name: provider cloud: nacos: config: server-addr: 127.0.0.1:8848 discovery: server-addr: 127.0.0.1:8848
启动类上添加 @EnableDiscoveryClient 注解
@EnableDiscoveryClient @SpringBootApplication public class ProviderApplication { public static void main(String[] args) { SpringApplication.run(ProviderApplication.class, args); } }
然后添加测试 controller
@RefreshScope @RestController @RequestMapping("/test") public class TestController { @Autowired private Environment env; @Value("${version:0}") private String version; /** * http://localhost:9001/test/port * @return */ @GetMapping("/port") public Object port() { return String.format("port=%s, version=%s", env.getProperty("local.server.port"), version); } }
注意,这里配置 nacos 的时候需要配置下面两个文件 provider.properties 和 provider,然后实际配置最终 nacos 是采用的 provider 文件,否则后端控制台就会持续输出 400 错误,可能是新版本问题,其他版本暂时不清楚(后面网关配置也是同理)
后端控制台输出,也可以看出需要两个
[fixed-localhost_8848] [subscribe] provider.properties+DEFAULT_GROUP [fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider.properties, group=DEFAULT_GROUP, cnt=1 [fixed-localhost_8848] [subscribe] provider+DEFAULT_GROUP [fixed-localhost_8848] [add-listener] ok, tenant=, dataId=provider, group=DEFAULT_GROUP, cnt=1
Nacos 版本:2.3.0-BETA
网关 gateway
gateway 服务的 pom 依赖配置如下:
org.springframework.cloud spring-cloud-starter-gateway org.projectlombok lombok true org.springframework.boot spring-boot-starter-test test org.junit.vintage junit-vintage-engine org.springframework.boot spring-boot-starter-actuator
application.yml
# 应用服务 WEB 访问端口 server: port: 9000 # 应用名称 spring: application: name: gateway cloud: nacos: config: server-addr: 127.0.0.1:8848 discovery: server-addr: 127.0.0.1:8848 gateway: routes: # http://127.0.0.1:9000/actuator/gateway/routes - id: provider # 路由 ID,保持唯一 uri: lb://provider # uri指目标服务地址,lb代表从注册中心获取服务 predicates: - Path=/provider/** # http://127.0.0.1:9000/provider/port 会转发到 http://localhost:9001/provider/port, 和预期不符合, 需要StripPrefix来处理 filters: - StripPrefix=1 # StripPrefix=1就代表截取路径的个数为1, 这样请求 http://127.0.0.1:9000/provider/test/port 会转发到 http://localhost:9001/test/port management: endpoint: gateway: enabled: true endpoints: web: exposure: include: gateway
同样启动类上添加 @EnableDiscoveryClient 注解
查看所有路由:/actuator/gateway/routes
查看指定路由(GET):/actuator/gateway/routes/{id}
查看全局过滤器:/actuator/gateway/globalfilters
查看路由过滤器:/actuator/gateway/routefilters
POST 方式刷新路由缓存:/actuator/gateway/refresh
测试
curl http://127.0.0.1:9001/test/port port=9001, version=2 curl http://127.0.0.1:9000/provider/test/port port=9001, version=2
动态路由
实现动态路由有两种方式,一个是改写 RouteDefinitionRepository(实测失败),一个是基于 nacos 的监听器给 RouteDefinitionRepository 动态更新值。实现逻辑大同小异
Spring Cloud Gateway 中加载路由信息分别由以下几个类负责
1、PropertiesRouteDefinitionLocator:从配置文件中读取路由信息 (如 YML、Properties 等)
2、RouteDefinitionRepository:从存储器中读取路由信息 (如内存、配置中心、Redis、MySQL 等)
3、DiscoveryClientRouteDefinitionLocator:从注册中心中读取路由信息(如 Nacos、Eurka、Zookeeper 等)
下面使用 RouteDefinitionRepository 配置动态路由
gateway-router.json
[{ "id": "provider", "predicates": [{ "name": "Path", "args": { "_genkey_0": "/provider/**" } }], "filters": [{ "name": "StripPrefix", "args": { "_genkey_0": "1" } }], "uri": "lb://provider", "order": 0 }]
NacosRouteDefinitionRepository 配置类
@Component public class NacosRouteDefinitionRepository implements RouteDefinitionRepository, ApplicationEventPublisherAware { private static final Logger log = LoggerFactory.getLogger(NacosRouteDefinitionRepository.class); @Autowired private NacosConfigManager nacosConfigManager; // 更新路由信息需要的 private ApplicationEventPublisher applicationEventPublisher; private String dataId = "gateway-router.json"; private String group = "DEFAULT_GROUP"; @Value("${spring.cloud.nacos.config.server-addr}") private String serverAddr; private ObjectMapper objectMapper = new ObjectMapper(); @PostConstruct public void dynamicRouteByNacosListener() { try { nacosConfigManager.getConfigService().addListener(dataId, group, new Listener() { public void receiveConfigInfo(String configInfo) { log.info("自动更新配置...\r\n{}", configInfo); applicationEventPublisher.publishEvent(new RefreshRoutesEvent(this)); } public Executor getExecutor() { return null; } }); } catch (NacosException e) { e.printStackTrace(); } } @Override public FluxgetRouteDefinitions() { try { String configInfo = nacosConfigManager.getConfigService().getConfig(dataId, group, 5000); List gatewayRouteDefinitions = objectMapper.readValue(configInfo, new TypeReference >() { }); return Flux.fromIterable(gatewayRouteDefinitions); } catch (NacosException e) { e.printStackTrace(); } catch (JsonMappingException e) { e.printStackTrace(); } catch (JsonProcessingException e) { e.printStackTrace(); } return Flux.fromIterable(Lists.newArrayList()); } @Override public Mono
save(Mono route) { return null; } @Override public Mono delete(Mono routeId) { return null; } @Override public void setApplicationEventPublisher(ApplicationEventPublisher applicationEventPublisher) { this.applicationEventPublisher = applicationEventPublisher; } }
然后重启网关,访问 http://127.0.0.1:9000/actuator/gateway/routes ,查看是否生效
[ { "predicate": "Paths: [/provider/**], match trailing slash: true", "route_id": "provider", "filters": [ "[[StripPrefix parts = 1], order = 1]" ], "uri": "lb://provider", "order": 0 } ]
灰度发布
首先需要明白灰度的场景, 因为有不同版本的服务需要共存, 所以新的节点升级的时候必然代码及配置会存在差别, 所以我们根据这种差别来判断服务版本是新版本还是线上稳定版本。这里我们用 prod 和 gray 来标识 2 个版本。
实现的整体思路:
- 编写带版本号的灰度路由(负载均衡策略)
- 编写自定义 filter
- nacos 服务配置需要灰度发布的服务的元数据信息以及权重(在服务 jar 中配置)
注意, 应该先修改 nacos 配置实现动态路由, 然后再升级灰度节点. 本案例只是简单示例灰度原理。
下面进行网关配置
依赖配置
首先排除掉默认的 ribbon 依赖
com.alibaba.cloud spring-cloud-starter-alibaba-nacos-discovery org.springframework.cloud spring-cloud-starter-netflix-ribbon
引入官方新的负载均衡包
org.springframework.cloud spring-cloud-starter-loadbalancer
负载均衡策略
public class VersionGrayLoadBalancer implements ReactorServiceInstanceLoadBalancer { private ObjectProviderserviceInstanceListSupplierProvider; private String serviceId; private final AtomicInteger position; public VersionGrayLoadBalancer(ObjectProvider serviceInstanceListSupplierProvider, String serviceId) { this(serviceInstanceListSupplierProvider, serviceId, new Random().nextInt(1000)); } public VersionGrayLoadBalancer(ObjectProvider serviceInstanceListSupplierProvider, String serviceId, int seedPosition) { this.serviceId = serviceId; this.serviceInstanceListSupplierProvider = serviceInstanceListSupplierProvider; this.position = new AtomicInteger(seedPosition); } @Override public Mono > choose(Request request) { HttpHeaders headers = (HttpHeaders) request.getContext(); ServiceInstanceListSupplier supplier = this.serviceInstanceListSupplierProvider.getIfAvailable(NoopServiceInstanceListSupplier::new); return ((Flux) supplier.get()).next().map(list -> processInstanceResponse((List ) list, headers)); } private Response processInstanceResponse(List instances, HttpHeaders headers) { if (instances.isEmpty()) { return new EmptyResponse(); } else { String reqVersion = headers.getFirst("version"); if (StringUtils.isEmpty(reqVersion)) { return processRibbonInstanceResponse(instances); } List serviceInstances = instances.stream() .filter(instance -> reqVersion.equals(instance.getMetadata().get("version"))) .collect(Collectors.toList()); if (serviceInstances.size() > 0) { return processRibbonInstanceResponse(serviceInstances); } else { return processRibbonInstanceResponse(instances); } } } /** * 负载均衡器 * 参考 org.springframework.cloud.loadbalancer.core.RoundRobinLoadBalancer#getInstanceResponse * * @author javadaily */ private Response processRibbonInstanceResponse(List instances) { int pos = Math.abs(this.position.incrementAndGet()); ServiceInstance instance = instances.get(pos % instances.size()); return new DefaultResponse(instance); } }
过滤器加载负载均衡
public class GrayReactiveLoadBalancerClientFilter implements GlobalFilter, Ordered { private static final Log log = LogFactory.getLog(ReactiveLoadBalancerClientFilter.class); private static final int LOAD_BALANCER_CLIENT_FILTER_ORDER = 10150; private final LoadBalancerClientFactory clientFactory; private LoadBalancerProperties properties; public GrayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) { this.clientFactory = clientFactory; this.properties = properties; } @Override public int getOrder() { return LOAD_BALANCER_CLIENT_FILTER_ORDER; } @Override public Monofilter(ServerWebExchange exchange, GatewayFilterChain chain) { URI url = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR); String schemePrefix = exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_SCHEME_PREFIX_ATTR); if (url != null && ("grayLb".equals(url.getScheme()) || "grayLb".equals(schemePrefix))) { ServerWebExchangeUtils.addOriginalRequestUrl(exchange, url); if (log.isTraceEnabled()) { log.trace(ReactiveLoadBalancerClientFilter.class.getSimpleName() + " url before: " + url); } return this.choose(exchange).doOnNext((response) -> { if (!response.hasServer()) { throw NotFoundException.create(this.properties.isUse404(), "Unable to find instance for " + url.getHost()); } else { URI uri = exchange.getRequest().getURI(); String overrideScheme = null; if (schemePrefix != null) { overrideScheme = url.getScheme(); } DelegatingServiceInstance serviceInstance = new DelegatingServiceInstance((ServiceInstance) response.getServer(), overrideScheme); URI requestUrl = this.reconstructURI(serviceInstance, uri); if (log.isTraceEnabled()) { log.trace("LoadBalancerClientFilter url chosen: " + requestUrl); } exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, requestUrl); } }).then(chain.filter(exchange)); } else { return chain.filter(exchange); } } protected URI reconstructURI(ServiceInstance serviceInstance, URI original) { return LoadBalancerUriTools.reconstructURI(serviceInstance, original); } private Mono > choose(ServerWebExchange exchange) { URI uri = (URI) exchange.getAttribute(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR); VersionGrayLoadBalancer loadBalancer = new VersionGrayLoadBalancer(clientFactory.getLazyProvider(uri.getHost(), ServiceInstanceListSupplier.class), uri.getHost()); if (loadBalancer == null) { throw new NotFoundException("No loadbalancer available for " + uri.getHost()); } else { return loadBalancer.choose(this.createRequest(exchange)); } } private Request createRequest(ServerWebExchange exchange) { HttpHeaders headers = exchange.getRequest().getHeaders(); Request request = new DefaultRequest<>(headers); return request; } }
注入过滤器
@Configuration public class GrayGatewayReactiveLoadBalancerClientAutoConfiguration { @Bean @ConditionalOnMissingBean({GrayReactiveLoadBalancerClientFilter.class}) public GrayReactiveLoadBalancerClientFilter grayReactiveLoadBalancerClientFilter(LoadBalancerClientFactory clientFactory, LoadBalancerProperties properties) { return new GrayReactiveLoadBalancerClientFilter(clientFactory, properties); } }
发布灰度服务
生产环境配置文件 application-prod.yml
server: port: 9002 spring: application: name: provider cloud: nacos: config: server-addr: 127.0.0.1:8848 discovery: metadata: version: prod server-addr: 127.0.0.1:8848
灰度环境配置文件 application-gray.yml
server: port: 9003 spring: application: name: provider cloud: nacos: config: server-addr: 127.0.0.1:8848 discovery: metadata: version: gray server-addr: 127.0.0.1:8848
idea 启动参数指定配置文件
同时注意配置两个环境的 nacos 文件(prod version: 4,gray version: 5)
测试
然后分别启动三个服务:9000 端口网关 gateway 服务,9002 端口生产环境 provider-prod 服务,9003 端口灰度环境 provider-gray 服务
E:\Nacos\nacos>curl http://127.0.0.1:9000/provider/test/port port=9003, version=5 E:\Nacos\nacos>curl -X GET -H "version:prod" http://127.0.0.1:9000/provider/test/port port=9003, version=5 E:\Nacos\nacos>curl -X GET -H "version:gray" http://127.0.0.1:9000/provider/test/port port=9002, version=4
仓库地址
如果对于文章中代码有疑问,可以直接查看下方作者仓库
仓库地址:ReturnTmp/spring-gateway-demo: 网关配置 + 灰度发布 + 配置中心 示例仓库 (github.com)
参考链接
- Spring Cloud Gateway+nacos灰度发布 - 掘金 (juejin.cn)
- SpringCloud gateway Actuator - shigp1 - 博客园 (cnblogs.com)
- Spring Cloud Gateway监控 周立的博客
- 前端灰度发布落地方案-腾讯云开发者社区-腾讯云 (tencent.com)
- 后端 - 灰度发布架构设计!值得品鉴
本文由博客一文多发平台 OpenWrite 发布!