OpenWhisk备忘录 2022-12-24 笔记,实验 暂无评论 1059 次阅读 [TOC] ## 用Helm部署OpenWhisk 最好用外网服务器。国内服务器直接看下一节。 ```bash sudo su mkdir ~/openwhisk cd ~/openwhisk # 安装docker apt install -y docker.io # 修改Docker镜像与cgroup驱动 sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": ["https://mirror.ccs.tencentyun.com"], "exec-opts": ["native.cgroupdriver=systemd"] } EOF sudo systemctl daemon-reload sudo systemctl restart docker # 安装KIND curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.17.0/kind-linux-amd64 chmod +x ./kind sudo mv ./kind /usr/local/bin/kind # 安装Kubectl curl -LO https://dl.k8s.io/release/v1.24.3/bin/linux/amd64/kubectl sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl # 安装helm curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash # 下载OpenWhisk部署 git clone https://github.com/apache/openwhisk-deploy-kube.git cd openwhisk-deploy-kube # 启动K8s集群 kind create cluster --wait=120s --config=./deploy/kind/kind-cluster.yaml # 部署OpenWhisk helm install owdev ./helm/openwhisk -n openwhisk --create-namespace -f ./deploy/kind/mycluster.yaml # 安装OpenWhisk的命令行工具 wsk cd ../ wget https://github.com/apache/openwhisk-cli/releases/download/1.2.0/OpenWhisk_CLI-1.2.0-linux-amd64.tgz tar zxvf OpenWhisk_CLI-1.2.0-linux-amd64.tgz mv ./wsk /usr/bin/ wsk property set --apihost localhost:31001 wsk property set --auth 23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP # 安装OpenWhisk部署工具 wskdeploy wget https://github.com/apache/openwhisk-wskdeploy/releases/download/1.2.0/openwhisk_wskdeploy-1.2.0-linux-amd64.tgz tar zxvf openwhisk_wskdeploy-1.2.0-linux-amd64.tgz mv ./wskdeploy /usr/bin/ ``` helm会把安装过程作为k8s的一个pod来执行，当此pod状态变成了Completed，说明OpenWhisk部署完成了。用以下命令查询此pod状态： ``` kubectl get pods -n openwhisk |grep install-packages ``` 等部署完成后，用`wsk list -v`来验证安装是否成功。 ## 从源码运行OpenWhisk - OpenWhisk的Standalone模式文档：https://github.com/apache/openwhisk/blob/master/core/standalone/README.md ```bash sudo su mkdir ~/openwhisk cd ~/openwhisk # 安装docker apt install -y docker.io # 修改Docker镜像与cgroup驱动 sudo mkdir -p /etc/docker sudo tee /etc/docker/daemon.json <<-'EOF' { "registry-mirrors": ["https://mirror.ccs.tencentyun.com"], "exec-opts": ["native.cgroupdriver=systemd"] } EOF sudo systemctl daemon-reload sudo systemctl restart docker # 安装OpenWhisk的命令行工具 wsk wget https://github.com/apache/openwhisk-cli/releases/download/1.2.0/OpenWhisk_CLI-1.2.0-linux-amd64.tgz mkdir ./wsk tar zxvf OpenWhisk_CLI-1.2.0-linux-amd64.tgz -C ./wsk mv ./wsk/wsk /usr/bin/ rm -rf ./wsk # 安装OpenWhisk部署工具 wskdeploy wget https://github.com/apache/openwhisk-wskdeploy/releases/download/1.2.0/openwhisk_wskdeploy-1.2.0-linux-amd64.tgz mkdir ./wskdeploy tar zxvf openwhisk_wskdeploy-1.2.0-linux-amd64.tgz -C ./wskdeploy mv ./wskdeploy/wskdeploy /usr/bin/ rm -rf ./wskdeploy wsk property set \ --apihost 'http://172.17.0.1:3233' \ --auth '23bc46b1-71f6-4ed5-8c54-816aa4f8c502:123zO3xZCLrMN6v2BKK1dXYFpXlPkccOFqm12CdAsMgRU4VrNZ9lyGVCGuMDGIwP' # 安装Java和NodeJS apt install -y openjdk-11-jdk nodejs npm # 下载OpenWhisk源码 git clone https://github.com/apache/openwhisk.git cd openwhisk # 构造（构造结果为./bin/openwhisk-standalone.jar） ./gradlew :core:standalone:build # 运行 java -jar bin/openwhisk-standalone.jar --all ``` > 下面是运行输出的一部分，是OpenWhisk启动的服务： ``` ================================================================================ Launched service details [ 3233 ] http://172.17.0.1:3233 (Controller) [ 6379 ] http://localhost:6379 (whisk-redis) [ 9000 ] http://localhost:9000 (whisk-apigw, Api Gateway - Api Service ) [ 3234 ] http://localhost:3234 (whisk-apigw, Api Gateway - Management Service) [ 5984 ] http://localhost:5984/_utils (whisk-couch, Username: [whisk_admin], Password: [some_passw0rd]) [ 9092 ] localhost:9092 (kafka) [ 9091 ] 172.17.0.1:9091 (kafka-docker) [ 2181 ] Zookeeper (zookeeper) [ 39039 ] http://localhost:39039 (whisk-kafka-drop-ui) [ 3235 ] http://localhost:3235 (whisk-user-events) [ 9090 ] http://localhost:9090 (whisk-prometheus) [ 3000 ] http://localhost:3000 (whisk-grafana) Local working directory - /root/.openwhisk/standalone/server-3233 ================================================================================ ``` 也是用`wsk list -v`来验证安装是否成功。 - 注意事项 - 从源码运行openwhisk无需k8s集群，不用安装kind、kubectl - 如果用kind创建k8s集群，不能将集群命名为openwhisk，不然openwhisk启动时会删掉该集群。 - 如果报错`[StandaloneOpenWhisk] Error starting Kafkajava.lang.IllegalArgumentException: requirement failed: ContainerIp must not be empty`，可能是上次运行未正常关闭，导致旧容器创建的文件夹没有删除，新容器无访问权限。解决方法是`rm -rf /root/.openwhisk/standalone/`。 ## 使用OpenWhisk ### 创建函数 ``` mkdir ~/openwhisk/example cd ~/openwhisk/example # 写函数代码 cat <<EOF >./hello.py def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting} EOF wsk action create helloPy hello.py --kind python:3 ``` > 运行结果： ``` root@k8smaster:~/openwhisk/example# wsk action create helloPy hello.py --kind python:3 ok: created action helloPy ``` ### 调用函数 ``` wsk action invoke helloPy --result --param name World ``` > 运行结果： ``` root@k8smaster:~/openwhisk/example# wsk action invoke helloPy --result --param name World { "greeting": "Hello World!" } ``` - 创建函数之前，Docker有的容器有： ``` root@k8smaster:~/openwhisk/openwhisk# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES e95cb03d170a openwhisk/user-events:nightly "./init.sh 0" 2 days ago Up 2 days 0.0.0.0:3235->9095/tcp, :::3235->9095/tcp whisk-user-events 0acd17e84d13 apache/couchdb:2.3 "tini -- /docker-ent…" 2 days ago Up 2 days 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp, :::5984->5984/tcp whisk-couch da3d27d82e9a obsidiandynamics/kafdrop "/kafdrop.sh" 2 days ago Up 2 days 0.0.0.0:40871->9000/tcp, :::40871->9000/tcp whisk-kafka-drop-ui 3ac447bb496f openwhisk/apigateway:0.11.0 "/usr/bin/dumb-init …" 2 days ago Up 2 days 80/tcp, 8423/tcp, 0.0.0.0:9000->9000/tcp, :::9000->9000/tcp, 0.0.0.0:3234->8080/tcp, :::3234->8080/tcp whisk-apigw ee620748dfaf redis:4.0 "docker-entrypoint.s…" 2 days ago Up 2 days 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp whisk-redis ``` - 创建（create）函数后，不会立刻为函数创建运行时容器，不过会创建一个预热容器。这个预热容器每隔10分钟就自动换成新的。 ``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES af77dc5b41ef openwhisk/action-nodejs-v14:nightly "docker-entrypoint.s…" 10 minutes ago Up 10 minutes 8080/tcp wsk0_3_prewarm_nodejs14 ``` - 在第一次调用（invoke），会花较长的时间创建容器，作为函数运行时。（冷启动） > ``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d9fecb295197 openwhisk/action-python-v3.7:nightly "/bin/proxy" 4 seconds ago Up 3 seconds wsk0_5_guest_helloPy 83b8278f8a87 openwhisk/action-nodejs-v14:nightly "docker-entrypoint.s…" 35 seconds ago Up 34 seconds 8080/tcp wsk0_4_prewarm_nodejs14 ``` - 紧接着几次调用属于热启动，响应很快。 - 一段时间不调用，容器变为Pause状态。 > ``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES d9fecb295197 openwhisk/action-python-v3.7:nightly "/bin/proxy" 4 minutes ago Up 4 minutes (Paused) wsk0_5_guest_helloPy 83b8278f8a87 openwhisk/action-nodejs-v14:nightly "docker-entrypoint.s…" 4 minutes ago Up 4 minutes 8080/tcp wsk0_4_prewarm_nodejs14 ``` - Pause状态下，函数调用仍是热启动，响应很快。运行时容器解除Pause状态，预热容器不变。 ``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 9d2386d09cce openwhisk/action-nodejs-v14:nightly "docker-entrypoint.s…" 3 minutes ago Up 3 minutes 8080/tcp wsk0_6_prewarm_nodejs14 d9fecb295197 openwhisk/action-python-v3.7:nightly "/bin/proxy" 12 minutes ago Up 12 minutes wsk0_5_guest_helloPy ``` - 第二天下午再看，运行时容器就已经自动删除了，预热容器还在。 ``` CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 65b787586d79 openwhisk/action-nodejs-v14:nightly "docker-entrypoint.s…" 8 minutes ago Up 8 minutes 8080/tcp wsk0_101_prewarm_nodejs14 ``` ### 创建两个python函数，看是否用同一个运行时容器 用wsk create方法创建helloA和helloB两个函数： ``` # 写函数代码 cat <<EOF >./hello_A.py def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": "A"} EOF cat <<EOF >./hello_B.py def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": "B"} EOF # 创建函数 wsk action create helloA hello_A.py --kind python:3 wsk action create helloB hello_B.py --kind python:3 ``` 用wskdeploy方法创建helloC和helloD两个函数： ``` # 写函数配置（含代码） cat <<EOF >./hello_CD.yaml packages: default: actions: helloC: code: | import sys def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": "C"} runtime: python:3 helloD: code: | import sys def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": "D"} runtime: python:3 EOF # 部署函数 wskdeploy -m hello_CD.yaml ``` >查看当前部署的函数 ``` root@k8smaster:/home/ubuntu/openwhisk/example# wsk list Entities in namespace: default packages actions /guest/helloD private python:3 /guest/helloC private python:3 /guest/helloB private python:3 /guest/helloA private python:3 /guest/helloPy private python:3 triggers rules ``` - 刚部署完，没调用，没有运行时容器，只有一个预热容器 - 调用helloA：创建helloA运行时容器，预热容器无变化。 - 调用helloB：创建helloB运行时容器，预热容器无变化，helloA容器变为pause状态 - 调用helloC：创建helloC运行时容器，预热容器无变化，helloA容器保持pause状态，helloB容器变为pause状态 - 调用helloD：创建helloD运行时容器，预热容器无变化，helloA容器被删除，helloB容器保持pause状态，helloC容器变为pause状态。 - 再调用helloPy、helloA、helloB、helloC、helloD，发现： - 运行时容器同时只有3个 - 运行时容器里上次调用10秒后，进入pause状态 - 只有服务器首次调用python3函数时需要拉镜像，之后所有python3函数都不需要拉镜像，冷启动很快。 - 运行时容器大概等30分钟无调用，就自动删除 ### 测一下含复杂库的启动时间 ``` cat <<EOF >./hello_scipy.py import scipy def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": f"scipy_{scipy.__version__}"} EOF # 创建函数 wsk action create helloSP hello_scipy.py --kind python:3 # 调用函数 wsk action invoke helloSP --result --param name World ``` >返回错误 ``` { "error": "Cannot start action. Check logs for details." } ``` ### 测一下运行路径 ``` cat <<EOF >./hello_pwd.py import os def main(dict): if 'name' in dict: name = dict['name'] else: name = "stranger" greeting = "Hello " + name + "!" print(greeting) return {"greeting": greeting, "func_name": f"pwd:{os.path.abspath('.')}"} EOF # 创建函数 wsk action create helloPWD hello_pwd.py --kind python:3 # 调用函数 wsk action invoke helloPWD --result --param name World ``` >执行路径是：/action/1/bin，说明运行环境在容器内。 ### 看代码 #### 参考文档 - Java文档（英文）：https://docs.oracle.com/en/java/javase/11/docs/api/index.html - Java文档（中文）：https://www.runoob.com/manual/jdk11api/index.html - Scala文档：https://docs.scala-lang.org/zh-cn/overviews/ #### 性能测试 - OpenWhisk自带的性能测试工具位于`tools/owperf/` - 用于测试延迟与吞吐量 - 记录时间戳： - BI (Before Invocation) - 客户端调用函数、或事件触发规则的时间戳 - TS（Trigger Start) - 规则被触发的时间戳 - AS (Action Start) - 函数开始时间戳 - AE (Action End) - 函数结束时间戳 - AI (After Invocation) - 函数结果返回到客户端的时间戳 - 延迟指标： - OEA (Overhead of Entering Action) = AS - BI - D 函数返回的执行时间 - AD (Action duration) = AE - AS - OER (Overhead of Executing Request) = AE - BI - D - TA (Trigger to Answer) = AS - TS - ORA (Overhead of Returning from Action) = AI - AE - RTT (Round Trip Time) = AI - BI - ORTT (Overhead of RTT) = RTT - D - 吞吐量指标： - Attempts = 客户端数量 × 客户调用函数次数 ÷ 时隙长度，也叫Arrival rate - Requests = （客户调用函数次数）或（规则调用函数次数+1） - Activation = 时隙内函数启动的次数 - Invocations = 时隙内函数成功执行的次数，也叫Service rate - 可以多线程测试，设置调用速率，来测试高并发的情况 - 注：该工具通过API调用OpenWhisk，会根据环境变量`http_proxy`、`https_proxy`和`proxy`走代理。所以如果代理关了，环境变量proxy没取消，就调用失败。 - 调用记录： >测试1个客户端调用100次函数，进行4轮。 `./owperf.sh -a action -w 1 -i 100 -r 4` 提取后的结果： ``` input.activity = action input.blocking = none input.delta = 200 input.iterations = 100 input.period = undefined input.ratio = 4 input.parameter_size = 1000 input.workers = 1 input.master_activity = undefined input.master_blocking = undefined input.master_delta = undefined input.warmup = 5 input.delay = 50 input.pp_delay = 60000 input.burst_timing = undefined input.setup = true input.teardown = false input.config_file = /root/.wskprops input.quiet = undefined input.master_apart = undefined output.measure_time = 20.023 output.ta.avg = undefined output.ta.std = undefined output.ta.min = undefined output.ta.max = undefined output.oea.avg = 233.958 output.oea.std = 278.322 output.oea.min = 11 output.oea.max = 901 output.oer.avg = 237.729 output.oer.std = 279.125 output.oer.min = 14 output.oer.max = 905 output.d.avg = 50.448 output.d.std = 0.748 output.d.min = 49 output.d.max = 54 output.ad.avg = 54.219 output.ad.std = 2.394 output.ad.min = 52 output.ad.max = 64 output.ora.avg = undefined output.ora.std = undefined output.ora.min = undefined output.ora.max = undefined output.rtt.avg = undefined output.rtt.std = undefined output.rtt.min = undefined output.rtt.max = undefined output.ortt.avg = undefined output.ortt.std = undefined output.ortt.min = undefined output.ortt.max = undefined output.attempts.abs = 400 output.attempts.tp = 19.977 output.attempts.tpw = 0 output.attempts.tpd = 100 output.invocations.abs = 96 output.invocations.tp = 4.794 output.invocations.tpw = 0 output.invocations.tpd = 100 output.activations.abs = 96 output.activations.tp = 4.794 output.activations.tpw = 0 output.activations.tpd = 100 output.requests.abs = 400 output.requests.tp = 19.977 output.requests.tpw = 0 output.requests.tpd = 100 output.errors.abs = 304 output.errors.percent = 76 ``` - 可以看到总共400次调用有304次出错。测试过程中OpenWhisk多次输出`[LocalEntitlementProvider] 'guest' has exceeded its throttle limit, Too many requests in the last minute (count: 392, allowed: 60).`，说明是被OpenWhisk限速了，只允许每分钟调用60次。 - 函数实际执行时间平均约50ms，而函数完成前的额外的时间开销平均约237ms。 - [neerajas-group/openwhisk-benchmarks](https://github.com/neerajas-group/openwhisk-benchmarks/blob/master/runtests/qr/run-qr.sh) - 用于测试延迟与内存占用 - 用命令行工具wsk操作OpenWhisk。 - 标签: none 本作品采用 知识共享署名-相同方式共享 4.0 国际许可协议 进行许可。
评论已关闭