Knative墙内安装

knative是一个大熔炉,将DevOps构建到服务的自动弹性伸缩,流量管控,事件驱动等都整合到一起。从很早期的版本开始我就一直有在关注,好在其文档还比较完备,可以基于文档一步步部署和尝鲜。

东西虽好,但安装却很麻烦!gcr.io一方面被墙,无法拉取镜像;另一方面gcr镜像名称还支持多级子目录,导致没法和docker hub简单做映射。在v0.4.0版本时,我曾一个个镜像在docker hub创建对应的repository。可是这货版本更新又很快,实在无法容忍;于是写了一个脚本,专门用于解决knative墙内安装的问题。

当前v0.6.0已经验证通过!

基本原理

其实原理比较简单,主要分两步:

  • 在墙外执行第一个脚本
    先基于knative release的yaml文件,从中过滤出gcr中存放的images;将其下载到墙外主机,然后按照特定规则更改每一个镜像的tag为满足docker hub中镜像名称规则的new tag,再推送到docker hub;
  • 在墙内执行第二个脚本
    按照第一个脚本中镜像old tag到new tag的转换规则,更新knative release的yaml文件中所有的镜像tag。然后使用更新后的yaml文件来部署knative。

安装istio

先在k8s上安装istio,具体操作如下(如果你说你不会安装k8s,请移步这里):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# Download and unpack Istio
export ISTIO_VERSION=1.1.3
curl -L https://git.io/getLatestIstio | sh -
cd istio-${ISTIO_VERSION}

for i in install/kubernetes/helm/istio-init/files/crd*yaml; do kubectl apply -f $i; done

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Namespace
metadata:
name: istio-system
labels:
istio-injection: disabled
EOF

helm template --namespace=istio-system \
--set prometheus.enabled=false \
--set mixer.enabled=false \
--set mixer.policy.enabled=false \
--set mixer.telemetry.enabled=false \
`# Pilot doesn't need a sidecar.` \
--set pilot.sidecar=false \
`# Disable galley (and things requiring galley).` \
--set galley.enabled=false \
--set global.useMCP=false \
`# Disable security / policy.` \
--set security.enabled=false \
--set global.disablePolicyChecks=true \
`# Disable sidecar injection.` \
--set sidecarInjectorWebhook.enabled=false \
--set global.proxy.autoInject=disabled \
--set global.omitSidecarInjectorConfigMap=true \
`# Set gateway pods to 1 to sidestep eventual consistency / readiness problems.` \
--set gateways.istio-ingressgateway.autoscaleMin=1 \
--set gateways.istio-ingressgateway.autoscaleMax=1 \
`# Set pilot trace sampling to 100%` \
--set pilot.traceSampling=100 \
install/kubernetes/helm/istio \
> ./istio-lean.yaml

kubectl apply -f istio-lean.yaml

生成安装所需yaml

这里也就是前面提到的第二个脚本文件,里面的REGISTRY_URL变量是写的我在docker hub的仓库名称。你也可以直接使用该脚本来部署,镜像都是public的。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
cat <<EOF > knative-image.sh
KNATIVE_PATH='knative'
KNATIVE_VER='v0.6.0'
REGISTRY_URL='ljchen'

rm -rf $KNATIVE_PATH
mkdir $KNATIVE_PATH


# download yaml

echo "download yaml files ..."

cd $KNATIVE_PATH

wget -q https://github.com/knative/serving/releases/download/${KNATIVE_VER}/serving.yaml
wget -q https://github.com/knative/build/releases/download/${KNATIVE_VER}/build.yaml
wget -q https://github.com/knative/eventing/releases/download/${KNATIVE_VER}/release.yaml
wget -q https://github.com/knative/eventing-sources/releases/download/${KNATIVE_VER}/eventing-sources.yaml
wget -q https://github.com/knative/serving/releases/download/${KNATIVE_VER}/monitoring.yaml
wget -q https://raw.githubusercontent.com/knative/serving/${KNATIVE_VER}/third_party/config/build/clusterrole.yaml

cd ..


# get images list

echo "collect image to tmp file ..."

cd $KNATIVE_PATH
rm -rf image.tmp

for line in `grep -RI " image: " *.yaml | grep gcr.io`
do
if [[ ${line} =~ 'gcr.io' ]]
then
if [[ ${line} =~ 'gcr.io/knative-releases/github.com/knative' ]]
then
sub_line1=${line##gcr.io/knative-releases/github.com/knative/}
sub_line2=${sub_line1%%@sha*}
container_name=knative_${sub_line2//\//_}

echo ${line#image:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp
else
sub_line1=${line#image:}
sub_line2=${sub_line1#*/}
sub_line3=${sub_line2%%:*}
container_name=knative_${sub_line3//\//_}

echo ${line#image:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp;
fi
fi
done

for line in `grep -RI " value: " *.yaml | grep gcr.io`
do
if [[ ${line} =~ 'gcr.io' ]]
then
if [[ ${line} =~ 'gcr.io/knative-releases/github.com/knative' ]]
then
sub_line1=${line##gcr.io/knative-releases/github.com/knative/}
sub_line2=${sub_line1%%@sha*}
container_name=knative_${sub_line2//\//_}

echo ${line#value:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp
else
sub_line1=${line#value:}
sub_line2=${sub_line1#*/}
sub_line3=${sub_line2%%:*}
container_name=knative_${sub_line3//\//_}

echo ${line#value:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp;
fi
fi
done


cd ..


# replace file image

cd $KNATIVE_PATH

counter=0
for file in *.yaml
do
echo "开始处理文件 " $file

while read line
do
origin_image=`echo ${line} | awk '{print $1}'`
new_image=`echo ${line} | awk '{print $2}'`

tmp=${origin_image//\//__}
origin_image=${tmp//__/\\/}

tmp2=${new_image//\//__}
new_image=${tmp2//__/\\/}

sed -i "s/${origin_image}/${new_image}/g" ${file}
#上面这行,如果是MacOS/UNIX请替换为: sed -i " " "s/${origin_image}/${new_image}/g" ${file}

done < image.tmp
counter=`expr ${counter} + 1`
done

echo "共处理文件数:" ${counter}

rm -rf *.yaml.1
cd ..


# finish

echo "completed..."
EOF

sh ./knative-image.sh

部署knative

这里就是用刚才生成的yaml文件来直接部署到knative到k8s了。

1
2
cd knative/
kubectl apply -f *.yaml

配套镜像生成脚本

这里就是前面说的第一个脚本,如果你想自己再跑一边,请修改REGISTRY_URL的值为你的镜像仓库地址。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101

KNATIVE_PATH='knative'
KNATIVE_VER='v0.6.0'
REGISTRY_URL='ljchen'

rm -rf $KNATIVE_PATH
mkdir $KNATIVE_PATH


# download yaml

echo "download yaml files ..."

cd $KNATIVE_PATH

wget -q https://github.com/knative/serving/releases/download/${KNATIVE_VER}/serving.yaml
wget -q https://github.com/knative/build/releases/download/${KNATIVE_VER}/build.yaml
wget -q https://github.com/knative/eventing/releases/download/${KNATIVE_VER}/release.yaml
wget -q https://github.com/knative/eventing-sources/releases/download/${KNATIVE_VER}/eventing-sources.yaml
wget -q https://github.com/knative/serving/releases/download/${KNATIVE_VER}/monitoring.yaml
wget -q https://raw.githubusercontent.com/knative/serving/${KNATIVE_VER}/third_party/config/build/clusterrole.yaml

cd ..


# get images list

echo "collect image to tmp file ..."

cd $KNATIVE_PATH
rm -rf image.tmp

for line in `grep -RI " image: " *.yaml | grep gcr.io`
do
if [[ ${line} =~ 'gcr.io' ]]
then
if [[ ${line} =~ 'gcr.io/knative-releases/github.com/knative' ]]
then
sub_line1=${line##gcr.io/knative-releases/github.com/knative/}
sub_line2=${sub_line1%%@sha*}
container_name=knative_${sub_line2//\//_}

echo ${line#image:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp
else
sub_line1=${line#image:}
sub_line2=${sub_line1#*/}
sub_line3=${sub_line2%%:*}
container_name=knative_${sub_line3//\//_}

echo ${line#image:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp;
fi
fi
done

for line in `grep -RI " value: " *.yaml | grep gcr.io`
do
if [[ ${line} =~ 'gcr.io' ]]
then
if [[ ${line} =~ 'gcr.io/knative-releases/github.com/knative' ]]
then
sub_line1=${line##gcr.io/knative-releases/github.com/knative/}
sub_line2=${sub_line1%%@sha*}
container_name=knative_${sub_line2//\//_}

echo ${line#value:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp
else
sub_line1=${line#value:}
sub_line2=${sub_line1#*/}
sub_line3=${sub_line2%%:*}
container_name=knative_${sub_line3//\//_}

echo ${line#value:} ${REGISTRY_URL}/${container_name}:${KNATIVE_VER} >> image.tmp;
fi
fi
done

cd ..


# download image, tag, push

cd $KNATIVE_PATH

while read line
do
origin_image=`echo ${line} | awk '{print $1}'`
new_image=`echo ${line} | awk '{print $2}'`

echo "old:" ${origin_image}
echo "new:" ${new_image}
docker pull ${origin_image}
docker tag ${origin_image} ${new_image}
docker push ${new_image}

done < image.tmp

cd ..

# done

echo "completed..."

gcr镜像映射表

下面排版不忍直视!尝试过表格等格式,发现更惨不忍睹。
hexo确实有点儿二;但是,内容的真实性更要紧,就忍忍吧~~

  1. gcr.io/knative-releases/github.com/knative/build/cmd/creds-init@sha256:101f537b53b895b28b84ac3c74ede7d250845e24c51c26516873d8ccb23168ce
    => ljchen/knative_build_cmd_creds-init:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/build/cmd/git-init@sha256:ce2c17308e9cb81992be153861c359a0c9e5f69c501a490633c8fe54ec992d53
    => ljchen/knative_build_cmd_git-init:v0.6.0
  1. gcr.io/cloud-builders/gcs-fetcher
    => ljchen/knative_cloud-builders_gcs-fetcher:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/build/cmd/nop@sha256:50e2be042298f24800b9840a9aef831a5fe4d89d9a8edea5e0559cdedf32369d
    => ljchen/knative_build_cmd_nop:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/build/cmd/controller@sha256:6a762848a46786cb481f5870787133e0d5e15615f8d54a5ba50d86b8315a58eb
    => ljchen/knative_build_cmd_controller:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/build/cmd/webhook@sha256:8f0bbc50b63f368c9959acab87838c6986691c28d424847459f3526bf97f8a3e
    => ljchen/knative_build_cmd_webhook:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/manager@sha256:99cf1f559f74ae97f271632697ed6e78a3fdd88a155632a57341b0dd6eab6581
    => ljchen/knative_eventing-sources_cmd_manager:v0.6.0
  1. k8s.gcr.io/elasticsearch:v5.6.4
    => ljchen/knative_elasticsearch:v0.6.0
  1. k8s.gcr.io/fluentd-elasticsearch:v2.0.4
    => ljchen/knative_fluentd-elasticsearch:v0.6.0
  1. k8s.gcr.io/addon-resizer:1.7
    => ljchen/knative_addon-resizer:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/controller@sha256:85c010633944c06f4c16253108c2338dba271971b2b5f2d877b8247fa19ff5cb
    => ljchen/knative_eventing_cmd_controller:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/sources_controller@sha256:aaa48f71a8db1b1dcf86c57d2dd72be1a65ed76d77f23a5abef4b2ad5c01c863
    => ljchen/knative_eventing_cmd_sources_controller:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/webhook@sha256:34a7cac96f8c809a7ce8ea0a86445204bbc6ac897525b876f53babb325f50bdc
    => ljchen/knative_eventing_cmd_webhook:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/in_memory/controller@sha256:496c19e81d9e7e40b3887c7c290304934f54f46c8a9186e800e314c014970c26
    => ljchen/knative_eventing_cmd_in_memory_controller:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/in_memory/dispatcher@sha256:897f03ed16e0000944da9ee0fdc971c43c8a494ff771c4e64d0573caf357c013
    => ljchen/knative_eventing_cmd_in_memory_dispatcher:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/queue@sha256:1e40c99ff5977daa2d69873fff604c6d09651af1f9ff15aadf8849b3ee77ab45
    => ljchen/knative_serving_cmd_queue:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/activator@sha256:f553b6cb7599f2f71190ddc93024952e22f2f55e97a3f38519d4d622fc751651
    => ljchen/knative_serving_cmd_activator:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/autoscaler@sha256:3a466eaf05cd505338163322331ee8634c601204250fa639360ae3524756acc3
    => ljchen/knative_serving_cmd_autoscaler:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/controller@sha256:8f402eab0ada038d3de2ad753a40f9f441715d08058d890537146bb0aba11c8e
    => ljchen/knative_serving_cmd_controller:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/networking/certmanager@sha256:dc77db09a23103f64a554de4e01cfda7371cbb13bc0954c991bdc4141169257f
    => ljchen/knative_serving_cmd_networking_certmanager:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/networking/istio@sha256:55fe9eeacfc20d97d3cd4f80bfc8a9b95cff7b5c50121bda87f754da8f05e57b
    => ljchen/knative_serving_cmd_networking_istio:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/serving/cmd/webhook@sha256:f0f98736bd4b55354f447f59183bf26b9be1ab01691b8b4aeee85caeb1166562
    => ljchen/knative_serving_cmd_webhook:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing-sources/cmd/github_receive_adapter@sha256:b5d6e12d16d16c6c42ae3d4325a1ef3a8a129dfc97740aa28000c0867edfc4ff
    => ljchen/knative_eventing-sources_cmd_github_receive_adapter:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/broker/ingress@sha256:a0acbe69420a67bef520e86aceaa237bf540c15882701c96245a6c4e06413bf6
    => ljchen/knative_eventing_cmd_broker_ingress:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/broker/filter@sha256:b4da7ce7b12aff2355066ed3237aadcf35df3b1c78db83cc538e6cffa564f208
    => ljchen/knative_eventing_cmd_broker_filter:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/cronjob_receive_adapter@sha256:6bbb724d5a4dbaaead890ea51d5f84eb9514974a2d06e26c8753db59010987fb
    => ljchen/knative_eventing_cmd_cronjob_receive_adapter:v0.6.0
  1. gcr.io/knative-releases/github.com/knative/eventing/cmd/apiserver_receive_adapter@sha256:7349f83eebe85a3eed7cdc4d442a935deab1ba0c42f34294f219f4ef17b59fec
    => ljchen/knative_eventing_cmd_apiserver_receive_adapter:v0.6.0
0%