Working with a Big Corporation

So it’s been a while since I started this job in a big corporation. I always enjoy new challenges, now my wish got granted. Not in a very good way.

The things work in a quite different manner here. There are big silos and layers between teams and departments, so the challenges here are not quite technical in nature. How unexpected this is.

Still there are lots of things can be improved with technology, here’s one example. When I was migrating an old web application stack from on-premises infrastructure to AWS, the AWS landing zone has already been provisioned with a duo-VPC setup. I really really miss the days that working with Kubernetes clusters and I can just run kubectl exec -ti ... and get a terminal session quickly.

Now things look like year 2000 and I need to use SSH proxy command again, without old school static IP addresses though. Ansible dynamic inventory is quite handy in most cases but it failed due to some unknown corporate firewall rules. I still have bash, aws-cli and jq, so this is my handy bash script to connect to 1 instance of an auto scaling group, via a bastion host(they both can be rebuilt and change IP).

#!/bin/bash
function get_stack_ip(){
aws ec2 describe-instances \
--fileter "Name=tag-key,Values=aws:cloudformation:stack-name" "Name=tag-value,Values=$1" \
|jq '.Reservations[] |select(.Instance[0].PrivateIpAddress != null).Instance[0].PrivateIpAddress' \
|tr -d '"'
}

Then it’s easy to use this function to get IPs of the bastion stack and the target stack, such as:

IP_BASTION=$(get_stack_ip bastion_stack)
IP_TARGET=$(get_stack_ip target_stack)
ssh -o ProxyCommand="ssh [email protected]_BASTION nc %h %p" [email protected]_TARGET

🙂

Don’t Need Ngrok When I Have SSH

I was trying to create a Slack app. In order to let Slack send REST requests to my dev environment, eg. http://localhost:9000, I searched a bit and saw ngrok. Ngrok is very handy for this kind of setup:

Slack -> xyz.ngrok.io -> localhost

However I just don’t want to install anything so I turned to Google and to my surprise SSH can exactly do this(for who knows how many years). I know I can forward a local port to a remote host to connect to a service behind firewall such as databases, this is my first attempt to forward a remote port to local so Slack API can contact my localhost.

Here’s a better article which explained how to do port forwarding in both directions with SSH.

In short, to forward a remote port to my localhost, I need to

1, update the sshd_config on remote host and have GatewayPorts enabled and then restart SSH service

GatewayPorts yes

2, in a local terminal, run the following command replacing my.remote.host with your server’s domain or IP.

ssh -nNT -R 9800:localhost:9000 my.remote.host

Then test it with

curl -i http://my.remote.host:9800

The request should be forwarded to your localhost:9000.

🙂

用 ssh_config 为 CLI 提速

最常用的命令, 应该是最简短的. 就好比常用的词句, 例如, 你好, 再见, 都是简短的. 惭愧的是, 我才想起来优化我的 CLI, 看来以前的工作压力还不够大 ^_^

参考(man) ssh_config, 可以把常用的 ssh 命令的参数写在 ~/.ssh/config 文件内. 最简单的格式是:

Host h1

HostName 10.0.0.100
User raymond

存盘后, 下次连接到 10.0.0.100, 只需要输入

ssh h1

就相当于

ssh [email protected]

一样了. 进一步的, 可以是:

Host h1

HostName 10.0.0.100
User raymond
Port 10022
ForwardAgent yes
ProxyCommand ssh [email protected] nc -w 1 %h 22

这样输入 ssh h1 就相当于

ssh -A -o "ProxyCommand ssh [email protected] nc -w 1 %h 22" -p 10022 [email protected]

感觉赚大发了. 另外可以输入如下命令来发现自己最常用的10条命令:

history | awk '{CMD[$2]++;count++;}END { for (a in CMD)print CMD[a] " " CMD[a]/count*100 "% " a;}' | grep -v "./" | column -c3 -s " " -t | sort -nr | nl | head -n10

🙂

让傀儡机去洗洗睡吧

DenyHosts在这里.

sshd:
Authentication Failures:
unknown (124.124.59.60): 8496 Time(s)
root (124.124.59.60): 1166 Time(s)
mail (124.124.59.60): 67 Time(s)
mysql (124.124.59.60): 67 Time(s)
nobody (124.124.59.60): 62 Time(s)
……

“我最讨厌你们这些劫匪了,一点技术含量都没有。” 当你看到某人或某傀儡机一遍一遍又一遍的蒙你的登录账号(俗称暴力破解), 你也会这么想吧.

Google了一下, 原来Centos已经包含了简单且有效的解决方案. 如果还没有安装DenyHosts可以一步安装:

sudo yum install denyhosts

缺省配置(/etc/denyhosts.conf)基本可用, 改一下接收报告的Email地址, 就启动吧:

sudo chkconfig –levels 2345 denyhosts on
sudo service denyhosts start

首次启动可能时间长一些, 因为要分析全部的日志文件. 之后, 和那些”别有用心”的肉机傀儡机说再见吧.

Added the following hosts to /etc/hosts.deny:

111.68.108.6 (111.68.108.6.pern.pk)
115.92.190.222 (unknown)
218.14.203.206 (unknown)
41.204.167.5 (unknown)
187.17.73.102 (187-17-73-102.whservidor.com)
59.50.43.234 (unknown)
119.147.105.247 (unknown)
174.142.111.44 (ip-174-142-111-44.static.privatedns.com)
123.125.127.132 (unknown)
203.126.53.110 (unknown)
202.198.8.54 (unknown)
222.236.46.222 (unknown)
124.124.59.60 (unknown)
118.219.234.163 (unknown)
218.29.203.4 (hn.kd.ny.adsl)
222.141.118.117 (hn.kd.ny.adsl)
173.234.224.16 (ns0.rlookuphost.com)
213.5.67.2 (hosted-by.altushost.com)
202.111.175.123 (unknown)
……