serial: use tty_port pointers in the core code