Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenWRT as client, No Traffic through Tunnel #549

Closed
Tracked by #624
WaterNS opened this issue May 14, 2017 · 12 comments
Closed
Tracked by #624

OpenWRT as client, No Traffic through Tunnel #549

WaterNS opened this issue May 14, 2017 · 12 comments

Comments

@WaterNS
Copy link

WaterNS commented May 14, 2017

OS / Environment

OSX 10.12.4 - runs algo scripts
DigitalOcean - algo node set up here

Clients:

  • Mac: OSX 10.12.4
  • OpenWRT
    • Hardware: Linksys ACS 1900
    • FW Version: davidc502 fw (latest)
      • Lede Reboot SNAPSHOT r4088-7296767 / LuCI Master (git-17.122.36998-758555d)
      • Kernel: 4.9.20
    • IPSec implementation: Strongswan

Summary of the problem

Algo appears to set up properly.
Testing with the Mac client (using the provided .mobileconfig) is a success: Whoer.net shows expected DigitalOcean IP.

Testing with OpenWRT client results in what appears to be a successful connection, but not traffic appears to be routed through the IPSec tunnel.

Steps to reproduce the behavior

(once working, possible Wiki entry for OpenWRT client?)

  • Reset OpenWRT client to firmware defaults (clean slate)

Setup Algo instance:

cd ~/Desktop/algo-master
python -m ensurepip --user
python -m pip install --user --upgrade virtualenv
python -m virtualenv env && source env/bin/activate && python -m pip install -U pip && python -m pip install -r requirements.txt
vim config.cfg #set users to only be "user1"

./algo

Prep the OpenWRT with required packages

ssh root@192.168.1.1

mkdir /etc/tmp
opkg update

opkg install strongswan \
strongswan-minimal \
strongswan-mod-openssl \
strongswan-mod-pem \
strongswan-mod-pkcs12 \
strongswan-mod-hmac \
strongswan-mod-pkcs7 \
strongswan-mod-random \
strongswan-mod-nonce \
strongswan-mod-aes \
strongswan-mod-gcm \
strongswan-mod-stroke \
strongswan-mod-pkcs8 \
strongswan-mod-x509 \
strongswan-mod-pubkey \
strongswan-mod-sha2 \
strongswan-mod-kernel-netlink \
strongswan-mod-revocation \
strongswan-mod-pgp \
strongswan-mod-socket-default \
kmod-crypto-gcm # This last one was missing and was critical

exit

Send over the algo generated bits:

scp ~/Desktop/algo-master/configs/*/pki/certs/user1.crt root@192.168.1.1:/etc/ipsec.d/certs
scp ~/Desktop/algo-master/configs/*/pki/private/user1.key root@192.168.1.1:/etc/ipsec.d/private
scp ~/Desktop/algo-master/configs/*/pki/cacert.pem root@192.168.1.1:/etc/ipsec.d/cacerts
scp ~/Desktop/algo-master/configs/*/ipsec_user1.conf root@192.168.1.1:/etc/tmp
scp ~/Desktop/algo-master/configs/*/ipsec_user1.secrets root@192.168.1.1:/etc/tmp

Configure OpenWRT client with Algo bits:

ssh root@192.168.1.1

# Backup  ipsec config
if [ -f /etc/ipsec.conf.original ]; then
  cp /etc/ipsec.conf.original /etc/ipsec.conf
else
  cp /etc/ipsec.conf /etc/ipsec.conf.original
fi

# ipsec.config: Inject lan-passthrough connection
tee -a /etc/ipsec.conf <<EOF
conn lan-passthrough
    leftsubnet=192.168.1.1/24 # Replace with your LAN subnet
    rightsubnet=192.168.1.1/24 # Replace with your LAN subnet
    authby=never # No authentication necessary
    type=pass # passthrough
    auto=route # no need to ipsec up lan-passthrough

EOF

# ipsec.config: inject algo connection
cat /etc/tmp/ipsec_user1.conf >> /etc/ipsec.conf
# ipsec.config: tweak algo connection to auto=start
sed -i -e 's/auto=add/auto=start/g' /etc/ipsec.conf

# Backup ipsec.secrets
if [ -f /etc/ipsec.secrets.original ]; then
  cp /etc/ipsec.secrets.original /etc/ipsec.secrets
else
  cp /etc/ipsec.secrets /etc/ipsec.secrets.original
fi

# ipsec.secrets: Inject algo secret
cat /etc/tmp/ipsec_user1.secrets >> /etc/ipsec.secrets

# Stop IPSec service
ipsec stop

# Start IPSec service - in console
ipsec start --nofork #for testing in console

# Start IPSec service as daemon
ipsec start
# Show IPSec status
ipsec statusall

The way of deployment (cloud or local)

Cloud (digitalocean)

Expected behavior

  • Traffic on router goes through Algo instance (ie curl ipinfo.io/ip on OpenWRT router show Algo instance IP)
  • Traffic on clients behind OpenWRT router would go through Algo instance (ie Whoer.net on a wireless phone would show Algo instance info)

Actual behavior

OpenWRT ssh session:

root@LEDE:~# curl ipinfo.io/ip
xxx.xxx.xxx.xxx #(my.local.isp.ip)

Computer connected to OpenWRT router:

localcomp:~ user$ curl ipinfo.io/ip
xxx.xxx.xxx.xxx #(my.local.isp.ip)

Full log

cat /etc/ipsec.conf

# ipsec.conf - strongSwan IPsec configuration file

# basic configuration

config setup
	# strictcrlpolicy=yes
	# uniqueids = no

# Add connections here.

# Sample VPN connections

#conn sample-self-signed
#      leftsubnet=10.1.0.0/16
#      leftcert=selfCert.der
#      leftsendcert=never
#      right=192.168.0.2
#      rightsubnet=10.2.0.0/16
#      rightcert=peerCert.der
#      auto=start

#conn sample-with-ca-cert
#      leftsubnet=10.1.0.0/16
#      leftcert=myCert.pem
#      right=192.168.0.2
#      rightsubnet=10.2.0.0/16
#      rightid="C=CH, O=Linux strongSwan CN=peer name"
#      auto=start
conn lan-passthrough
    leftsubnet=192.168.1.1/24 # Replace with your LAN subnet
    rightsubnet=192.168.1.1/24 # Replace with your LAN subnet
    authby=never # No authentication necessary
    type=pass # passthrough
    auto=route # no need to ipsec up lan-passthrough

conn ikev2-107.170.197.223
    fragmentation=yes
    rekey=no
    dpdaction=clear
    keyexchange=ikev2
    compress=no
    dpddelay=35s

    ike=aes128gcm16-prfsha512-ecp256!
    esp=aes128gcm16-ecp256!

    right=107.170.197.223
    rightid=107.170.197.223
    rightsubnet=0.0.0.0/0
    rightauth=pubkey

    leftsourceip=%config
    leftauth=pubkey
    leftcert=user1.crt
    leftfirewall=yes
    left=%defaultroute

    auto=start

ipsec start --nofork

no files found matching '/etc/strongswan.d/*.conf'
Starting strongSwan 5.5.2 IPsec [starter]...
00[LIB] no files found matching '/etc/strongswan.d/*.conf'
00[DMN] Starting IKE charon daemon (strongSwan 5.5.2, Linux 4.9.20, armv7l)
00[CFG] loading ca certificates from '/etc/ipsec.d/cacerts'
00[CFG]   loaded ca certificate "CN=107.170.197.223" from '/etc/ipsec.d/cacerts/cacert.pem'
00[CFG] loading aa certificates from '/etc/ipsec.d/aacerts'
00[CFG] loading ocsp signer certificates from '/etc/ipsec.d/ocspcerts'
00[CFG] loading attribute certificates from '/etc/ipsec.d/acerts'
00[CFG] loading crls from '/etc/ipsec.d/crls'
00[CFG] loading secrets from '/etc/ipsec.secrets'
00[CFG]   loaded ECDSA private key from '/etc/ipsec.d/private/user1.key'
00[LIB] loaded plugins: charon aes sha2 sha1 random nonce x509 revocation pubkey pkcs7 pkcs8 pkcs12 pgp pem openssl gmp xcbc hmac gcm kernel-netlink socket-default stroke updown
00[JOB] spawning 16 worker threads
charon (7329) started after 180 ms
04[CFG] received stroke: add connection 'lan-passthrough'
04[CFG] added configuration 'lan-passthrough'
06[CFG] received stroke: route 'lan-passthrough'
'lan-passthrough' shunt PASS policy installed

10[CFG] received stroke: add connection 'ikev2-107.170.197.223'
10[CFG]   loaded certificate "CN=user1" from 'user1.crt'
10[CFG]   id '%any' not confirmed by certificate, defaulting to 'CN=user1'
10[CFG] added configuration 'ikev2-107.170.197.223'
11[CFG] received stroke: initiate 'ikev2-107.170.197.223'
11[IKE] initiating IKE_SA ikev2-107.170.197.223[1] to 107.170.197.223
11[ENC] generating IKE_SA_INIT request 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) N(FRAG_SUP) N(HASH_ALG) N(REDIR_SUP) ]
11[NET] sending packet: from my.local.isp.ip[500] to 107.170.197.223[500] (264 bytes)
13[NET] received packet: from 107.170.197.223[500] to my.local.isp.ip[500] (289 bytes)
13[ENC] parsed IKE_SA_INIT response 0 [ SA KE No N(NATD_S_IP) N(NATD_D_IP) CERTREQ N(FRAG_SUP) N(HASH_ALG) N(MULT_AUTH) ]
13[IKE] received cert request for "CN=107.170.197.223"
13[IKE] sending cert request for "CN=107.170.197.223"
13[IKE] authentication of 'CN=user1' (myself) with ECDSA_WITH_SHA256_DER successful
13[IKE] sending end entity cert "CN=user1"
13[IKE] establishing CHILD_SA ikev2-107.170.197.223
13[ENC] generating IKE_AUTH request 1 [ IDi CERT N(INIT_CONTACT) CERTREQ IDr AUTH CPRQ(ADDR) SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_6_ADDR) N(MULT_AUTH) N(EAP_ONLY) N(MSG_ID_SYN_SUP) ]
13[NET] sending packet: from my.local.isp.ip[4500] to 107.170.197.223[4500] (866 bytes)
14[NET] received packet: from 107.170.197.223[4500] to my.local.isp.ip[4500] (830 bytes)
14[ENC] parsed IKE_AUTH response 1 [ IDr CERT AUTH CPRP(ADDR DNS DNS) SA TSi TSr N(MOBIKE_SUP) N(ADD_4_ADDR) N(ADD_6_ADDR) ]
14[IKE] received end entity cert "CN=107.170.197.223"
14[CFG]   using certificate "CN=107.170.197.223"
14[CFG]   using trusted ca certificate "CN=107.170.197.223"
14[CFG] checking certificate status of "CN=107.170.197.223"
14[CFG] certificate status is not available
14[CFG]   reached self-signed root ca with a path length of 0
14[IKE] authentication of '107.170.197.223' with ECDSA_WITH_SHA256_DER successful
14[IKE] IKE_SA ikev2-107.170.197.223[1] established between my.local.isp.ip[CN=user1]...107.170.197.223[107.170.197.223]
14[CFG] handling INTERNAL_IP4_DNS attribute failed
14[CFG] handling INTERNAL_IP4_DNS attribute failed
14[IKE] installing new virtual IP 10.19.48.1
14[IKE] CHILD_SA ikev2-107.170.197.223{1} established with SPIs cb95b264_i ce815a1c_o and TS 10.19.48.1/32 === 0.0.0.0/0
14[IKE] peer supports MOBIKE
09[NET] received packet: from 107.170.197.223[4500] to my.local.isp.ip[4500] (57 bytes)
09[ENC] parsed INFORMATIONAL request 0 [ ]
09[ENC] generating INFORMATIONAL response 0 [ ]
09[NET] sending packet: from my.local.isp.ip[4500] to 107.170.197.223[4500] (57 bytes)

ipsec start

no files found matching '/etc/strongswan.d/*.conf'
Starting strongSwan 5.5.2 IPsec [starter]...

ipsec statusall

no files found matching '/etc/strongswan.d/*.conf'
Status of IKE charon daemon (strongSwan 5.5.2, Linux 4.9.20, armv7l):
  uptime: 49 seconds, since May 14 19:16:24 2017
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 1
  loaded plugins: charon aes sha2 sha1 random nonce x509 revocation pubkey pkcs7 pkcs8 pkcs12 pgp pem openssl gmp xcbc hmac gcm kernel-netlink socket-default stroke updown
Listening IP addresses:
  192.168.1.1
  fd67:f749:cf0b::1
  my.local.isp.ip
Connections:
lan-passthrough:  %any...%any  IKEv1/2
lan-passthrough:   local:  uses public key authentication
lan-passthrough:   remote: uses public key authentication
lan-passthrough:   child:  192.168.1.0/24 === 192.168.1.0/24 PASS
ikev2-107.170.197.223:  %any...107.170.197.223  IKEv2, dpddelay=35s
ikev2-107.170.197.223:   local:  [CN=user1] uses public key authentication
ikev2-107.170.197.223:    cert:  "CN=user1"
ikev2-107.170.197.223:   remote: [107.170.197.223] uses public key authentication
ikev2-107.170.197.223:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=clear
Shunted Connections:
lan-passthrough:  192.168.1.0/24 === 192.168.1.0/24 PASS
Security Associations (1 up, 0 connecting):
ikev2-107.170.197.223[1]: ESTABLISHED 48 seconds ago, my.local.isp.ip[CN=user1]...107.170.197.223[107.170.197.223]
ikev2-107.170.197.223[1]: IKEv2 SPIs: 951a6e81f35deadb_i* 8d7607b9e991c6c9_r, rekeying disabled
ikev2-107.170.197.223[1]: IKE proposal: AES_GCM_16_128/PRF_HMAC_SHA2_512/ECP_256
ikev2-107.170.197.223{1}:  INSTALLED, TUNNEL, reqid 1, ESP SPIs: cb858021_i c985d905_o
ikev2-107.170.197.223{1}:  AES_GCM_16_128, 0 bytes_i, 0 bytes_o, rekeying disabled
ikev2-107.170.197.223{1}:   10.19.48.1/32 === 0.0.0.0/0
root@LEDE:~# 
@Thermi
Copy link

Thermi commented May 15, 2017

OpenWRT is a mess. You need to fix your NAT rules in *nat POSTROUTING and SNAT the connections to the assigned virtual IP. You also need to configure the updown script to set up the SNAT and tear it down. The configuration itself is also a mess and the tunnel fails open. The proper way to do it is to use auto=route, but that doesn't work with "virtual" IPs yet.
left=%firewall doesn't do anything, rekey=no is detremental to the security of your VPN,

@WaterNS
Copy link
Author

WaterNS commented May 15, 2017

Oh man, didn't realize OpenWRT was considered client unsupported, actually thought it was the defacto test device/case...

Any recommendations on a device that could be used to place between router and connection as a VPN egress? Alternatively, any recommendations of VPN that could run nicely on OpenWRT?

It would be nice to see it working on OpenWRT but I currently do not understand enough about the postrouting, SNAT, and updown script to properly put together a fix (unless someone has some decent pointers in that regard).

@kiratp
Copy link

kiratp commented May 21, 2017

I would suggest buy an EdgeRouter and use this PR as a starting point. The actual changes I made in there are pretty straightforward. This will be the the easiest way for whole-network IPSEC at this point.

#473 (review)

@Thermi
Copy link

Thermi commented May 21, 2017 via email

@kiratp
Copy link

kiratp commented May 21, 2017

@Thermi - if anything - I had to disable SNAT for LAN clients to route over the tunnel. EdgeOS brings up the tunnel and keeps it up. Also I am using auto=route.

VirtualIP support is still lacking though.

What other products would you suggest to solve this scenario in a reasonable price range?

@Thermi
Copy link

Thermi commented May 21, 2017 via email

@kiratp
Copy link

kiratp commented May 21, 2017

The EdgeRouter X is $50 and the EdgeRouter Lite is $100. Both support IPsec HW offload (100s of Mb/s throughput) for AES256+sha128/256 - assuming you are willing to downgrade ciphers from GCM.

https://help.ubnt.com/hc/en-us/articles/115006567467-EdgeRouter-Hardware-Offloading-Explained

The other advantage over OpenWRT is that you get updates on a regular basis as opposed to having to wait for a maintainer to release another build for your particular router.

I have no affiliation with Ubiquity Networks - I just think they make a great product for the price.

@Thermi
Copy link

Thermi commented May 21, 2017 via email

@kiratp
Copy link

kiratp commented May 21, 2017

I don't want to get in to the Free vs proprietary debate here. For me, compiling my router firmware is a bridge too far. At that point I'd just deploy a pfsense box but that is a whole other project.

IMO, Ubiquiti, being one of the fastest growing SMB/rural ISP networking vendors, is incentivized enough to give me well tested updates, promptly. I am not dependent on their goodwill per se.

@WaterNS, if you want something working now, EdgeRouter X/Lite are 2 "appliance" options - you have to chose between the points @Thermi and I are debating.

Re: Ciphers - In my case, my primary concern is my keeping my ISP out so AES-CBC + HMAC-SHA256 is good enough. Its a fair tradeoff for me to actually get > 150Mb/s IPSec throughput (vs 30 Mb/s without HW offload). Each client on my LAN that supports GCM tunnels is given separate a NATed route out to maintain its own direct tunnel to Algo. The router tunnel is for all other the non-IPSec HW in my house like my IOT devices, media streamers etc.

sha128 - yeah I meant sha1 - flubbed it.

@WaterNS
Copy link
Author

WaterNS commented May 22, 2017

So I do like the idea of being able to purchase something like an EdgeRouter X/Lite and have a fairly well "supported" hardware device. I've ran my own PfSense, Monowall, unTangle, etc etc etc - there is absolutely something to be said about somewhat simplified (albeit often with limitations) but functioning solution - all the formers felt like running Hackintosh vs the real thing (sneeze wrong way in its direction beyond light weight load and it has a mini heart attack).

Things like TomatoUSB, OpenWRT builds maintained by community members, have been a happy medium with regards to unlocking more advanced capabilities but NOT forcing one to delve into cross compiling firmwares and debugging. To Thermi's point, OpenWRT actually has a somewhat decent package system that doesn't force someone create own packages/firmwares and adding packages can be done via opkg install x y z, but should there be a need, a new firmware/package can be whipped up and fairly easily installed.

Where it (currently) begins to fall apart for Edge Router at the moment is:

  1. Not (yet) fully/well supported configuration by Algo. I see PR Adding support for EdgeMax clients #473 is actively working on addressing that and I absolutely applaud that as I feel supporting Ubiquity implementation is a solid target.

  2. Hardware firmware doesn't support highest level standard requested by soft. If dropping $50-100 on what effectively because single use appliance, would like to see it have supported capability to max (i.e. AES-GCM). I do feel its a valid point that not being able to support the standards requested by Algo means might as well switch to sometimes better supported/documented OpenVPN implementation (unless I am mistaken).

  3. With the Ubiquity line, the proper way is to then add Ubiquity wifi access points (I reckon renders my Linksys ACS1900 to the parts/resale bin). Thats additional ~$300+ for a nice AC wifi access point. I do absolutely feel this would result in a more capable and extendable Wifi network but going back to it, means dropping maybe ~$400-500 on upgrading network. Doable, and I'd reckon VPN benefits enriches the value proposition, but its still hard to justify that sort of outlay for one main feature increase.

  4. EdgeOS(?) appears to be a proprietary platform. I'm not saying its not attainable ($50-$100 is a feasible bar), but to point # 2, ability to use the desired standards starts to be problematic when the maintainer doesn't current have a way to add them.

Whats not clear to me is what exactly is the defacto supported client device implementation. Is it a raspberry Pi running ubuntu variant? An Apple device? So far, the Apple devices seem to have the top level of support - the generated mobileconfigs seem to work a charm (and much props for that). I had thought OpenWRT was the defacto ideal client based on the readme and everything else was trying to mimic what is achievable there but turns out to not be current case.

Hopefully, none of the above is taken as criticism of either thoughts - There has been a lot of work and effort contributed and I do not wish any of my words to be taken in a way that would discourage furthering the journey. My hope is what can come is a well support option for whole network scenario. I'm thinking OpenWRT might be the answer here, but I will gladly consider Edge Router if the firmware supports latest/highest standards.

@Thermi
Copy link

Thermi commented May 22, 2017

OpenWRT IS the answer here, but you'll have to fix some firewall rules there to make it work (and preferably stop using LUCI to manage the firewall, because it's not aware of IPsec and tunnels).

@dguido dguido closed this as completed Jul 16, 2017
@dguido dguido mentioned this issue Jul 16, 2017
4 tasks
@haltunbay
Copy link
Contributor

On server side I have Algo on Digital Ocean and client side OpenWrt on TpLink. I couldn't make it work with strongswan but it worked like a charm with wireguard. See https://openwrt.org/docs/guide-user/services/vpn/wireguard/client

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants