Disable TLS 1.3 for ocserv's segfault workaround

by

ocserv is an open-source VPN software compatible to many proprietary solutions including Cisco AnyConnect, so I use it for my iOS devices.

Recently I noticed that my ocserv service was not working after upgrading to Ubuntu 22.04, and got this message:

[Tue Sep 26 17:35:27 2023] traps: ocserv-worker[6826] general protection fault ip:7f09bab28898 sp:7ffd36c9add0 error:0 in libc.so.6[7f09bab28000+195000]

It looked not a good sign for a security software.

After some trials, I found it's easy to reproduce the message via openssl s_client -connect vpn.example.com:1234 from another Ubuntu server.

So next was trying to catch its stack with debug information before segfault. I set up Debug Symbol Packages and then installed all related dbg packages, then run gdb with set follow-fork-mode child to catch issues including the ones in child processes.

Just few seconds, I got one:

(gdb) bt
#0  __GI_abort () at ./stdlib/abort.c:107
#1  0x00007fe81996545c in __libc_message (action=do_abort, fmt=0x7fe819ab77b1 "%s", 
    fmt=0x7fe819ab77b1 "%s", action=do_abort) at ../sysdeps/posix/libc_fatal.c:155
#2  0x00007fe819965770 in __GI___libc_fatal (
    message=message@entry=0x7fe819ab9d28 "The futex facility returned an unexpected error code.\n")
    at ../sysdeps/posix/libc_fatal.c:164
#3  0x00007fe819975f1a in futex_fatal_error () at ../sysdeps/nptl/futex-internal.h:87
#4  futex_wait (private=<optimized out>, expected=<optimized out>, futex_word=<optimized out>)
    at ../sysdeps/nptl/futex-internal.h:162
#5  futex_wait_simple (private=<optimized out>, expected=<optimized out>, futex_word=<optimized out>)
    at ../sysdeps/nptl/futex-internal.h:177
#6  __pthread_once_slow (once_control=0x7fe819eb5d88 <keylog_once>, 
    init_routine=0x7fe819d17020 <keylog_once_init>) at ./nptl/pthread_once.c:105
#7  0x00007fe819d1ea3d in gnutls_once (once=0x7fe819eb5d88 <keylog_once>, 
    init_func=0x7fe819d17020 <keylog_once_init>) at ../../lib/locks.c:115
#8  _gnutls_nss_keylog_write (secret_size=48, 
    secret=0x55de435c717c "x", label=0x7fe819e36af8 "CLIENT_HANDSHAKE_TRAFFIC_SECRET", 
    session=0x55de435c5970) at ../../lib/kx.c:158
#9  _gnutls_nss_keylog_func (secret=<optimized out>, secret=<optimized out>, 
    label=0x7fe819e36af8 "CLIENT_HANDSHAKE_TRAFFIC_SECRET", session=0x55de435c5970) at ../../lib/kx.c:131
#10 _gnutls_nss_keylog_func (session=0x55de435c5970, 
    label=0x7fe819e36af8 "CLIENT_HANDSHAKE_TRAFFIC_SECRET", secret=<optimized out>) at ../../lib/kx.c:121
#11 0x00007fe819d3c824 in _gnutls_call_keylog_func (size=<optimized out>, 
    data=0x55de435c717c "x", label=0x7fe819e36af8 "CLIENT_HANDSHAKE_TRAFFIC_SECRET", 
    session=0x55de435c5970) at ../../lib/kx.c:115
#12 _tls13_set_keys (key_size=32, iv_size=12, params=0x55de435c0e70, stage=STAGE_HS, 
    session=0x55de435c5970) at ../../lib/constate.c:438
#13 _gnutls_epoch_set_keys (session=session@entry=0x55de435c5970, epoch=epoch@entry=1, 
    stage=stage@entry=STAGE_HS) at ../../lib/constate.c:714
#14 0x00007fe819d3dcc0 in _tls13_connection_state_init (session=0x55de435c5970, stage=STAGE_HS)
    at ../../lib/constate.c:1208
#15 0x00007fe819d070e5 in _gnutls13_handshake_server (session=0x55de435c5970)
    at ../../lib/handshake-tls13.c:423
#16 0x00007fe819d1bda0 in gnutls_handshake (session=0x55de435c5970) at ../../lib/handshake.c:2874
#17 0x000055de422ea402 in vpn_server (ws=0x55de435aec20) at ./src/worker-vpn.c:859
#18 main (argc=<optimized out>, argv=<optimized out>) at ./src/worker.c:189

There was a _tls13_set_keys() called so I immediately gave it a try to see whether TLS 1.2 was working:

openssl s_client -tls1_2 -connect vpn.example.com:1234

And yes, it's working now.

So back to trace the bug and I saw gnutls_once() was called in ../../lib/locks.c:115, which you can see in locks.c's L115. But it's actaully completely rewritten in the next version 3.7.4.

Anyway, I needed a workaround allowing clients able to connect to my VPN server. The workaround I choose is to disable TLS 1.3, so it will not touch the problematic code path. Added -VERS-TLS1.3 to tls-priorities and everything goes back to normal:

tls-priorities = "NORMAL:%SERVER_PRECEDENCE:%COMPAT:-RSA:-VERS-SSL3.0:-ARCFOUR-128:-VERS-TLS1.3"

It's not optimal, but okay for now.

Stream logs to Slack in Perl

by

I want to stream my logs on my home RPi3b to a Slack channel for my vacation side project. With some searches I found an article including a simple shell script to implement this function: "Stream Any Log File to Slack Using curl".

However it's a little tricky as the script replaces " to ' as escaping mechanism. I would prefer to use more formal way to implement this, also, I haven't written Perl for a while, so I have decided to implement this function in Perl:

In this script there are some notes:

  • Choosing tail -F instead of File::Tail is because File::Tail is quite old and no inotify implementation. On the other side, tail performs well in this section.
  • Choosing JSON::PP (pure perl version) is because it's built-in in Perl 5.14+, and that's also why I have added use v5.14 requirement in the beginning.
  • Choosing WWW::Mechanize for API calling is because there is no built-in module able to handle HTTPS connections, therefore I have decided to install a familiar one (for me).

btw, // operator has been introduced in Perl 5.10, and I didn't think it last there for such a long time...

The weird part of AWS bandwidth cost

by

I just got curious during handling the invoice of AWS. My gut feeling is that all inbound traffic are free (mostly), for example, you can see this in "Amazon EC2 On-Demand Pricing":

Data Transfer IN To Amazon EC2 From Internet
All data transfer in $0.00 per GB

But if you're dealing with inbound traffic across AZ in the same region:

Data transferred "in" to and "out" from Amazon EC2, Amazon RDS, Amazon Redshift, Amazon DynamoDB Accelerator (DAX), and Amazon ElastiCache instances, Elastic Network Interfaces or VPC Peering connections across Availability Zones in the same AWS Region is charged at $0.01/GB in each direction.

Yeah, inbound traffic across AZ is more expensive than the one from internet (which is free).

Also, inbound traffic to public IPv4 addresses and/or Elastic IP are not free:

IPv4: Data transferred “in” to and “out” from public or Elastic IPv4 address is charged at $0.01/GB in each direction.
IPv6: Data transferred “in” to and “out” from an IPv6 address in a different VPC is charged at $0.01/GB in each direction.

Anyway, this reminds me that I need to count US$0.02/GB instead of US$0.01/GB for such sort of traffic.

Golang software packaging for offline environment

by

Open build services like Launchpad PPA usually forbid internet connections during building, therefore it's impossible to download dependencies during building, and we need to include all dependencies into the source package.

Two things need to be done for building Golang softwares on no-internet building services:

  • Download all dependencies into vendor/ via go mod vendor command. This is usually done on the local machine with internet connections.
  • Ask Golang compiler to use vendor/ via go build -mod vendor command. Usually inside debian/rules file or Makefile.

If you're also working on Launchpad PPA like me, it's also suggested that add Simon Eisenmann's Golang Backports to the building dependency settings. This repository includes new Golang softwares so that you won't be hit by something like "syntax error".