Sometimes you want to see functions of a library, as they’re called. I know of two ways of doing this.
Let’s have a super simple test program:
#include<iostream>
#include<unistd.h>
void func1() {}
void func2() {}
int main()
{
std::cout << "Hello world\n";
func1();
func2();
func1();
// Wait a bit for bpftrace to be able to aquire the function name.
// Not applicable for something that doesn't exist.
sleep(1);
}
Start a bpftrace in one terminal, and run the program in another.
$ sudo bpftrace -e 'uprobe:./a.out:func* { print(func); }'
Attaching 2 probes...
func1()
func2()
func1()
$ gdb a.out
[…]
(gdb) rbreak func.*
[…]
(gdb) commands
Type commands for breakpoint(s) 1-3, one per line.
End with a line saying just "end".
>silent
>bt 1
>cont
>end
(gdb) r
Starting program: […]/a.out
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
#0 0x0000555555555215 in _GLOBAL__sub_I__Z5func1v ()
Hello world
#0 0x000055555555516d in func1() ()
#0 0x0000555555555174 in func2() ()
#0 0x000055555555516d in func1() ()
[Inferior 1 (process 424744) exited normally]
(gdb)
bpftrace is lower (but Continue reading
I’ve been getting into RISC-V optimization recently. Partly because I got my SiFive VisionFive 2, and partly because unlike x86 the number of RISC-V instructions is so managable that I may actually have a chance at beating the compiler.
I’m optimizing the inner loops of GNURadio, or in other words the volk library. I’ve been getting up to a about a doubling of the speed compared to the compiled C code, depending on the function.
But it got me thinking how far I could tweak the compiler and its options, too.
Yes, I should have done this much sooner.
Many years ago now I built some data processing thing in C++, and thought it ran too slowly. Sure, I did a debug build, but how much slower could that be? Half speed? Nope. 20x slower.
Of course this time I never compared to a debug build, so don’t expect that kind of difference. Don’t expect that it’ll reach my hand optimized assembly either, imperfect as it may be.
This may look like a synthetic benchmark, in simplified C++:
complex volk_32fc_x2_dot_prod_32fc_generic(const vector<complex> &in1,
const vector<complex> &in2)
{
complex res;
for (unsigned int i = 0; i Continue readingFor a long time I’ve wanted something Raspberry-pi-like but with RISC-V. And finally there is one, and a defensible price! Especially with the Raspberry Pi 4 shortage this seemed like a good idea.
This post is my first impressions and setup steps.
When I was in my late teens I was playing with different architectures, mostly using discarded university computers. It was fun to have such different types of computers. Back then it was SPARC (And UltraSparc), Alpha, and x86. Maybe access to some HPPA. I even had a MIPS (SGI Indigo 2).
Nowadays instead of SPARC, Alpha, and x86 it’s ARM, RISC-V, and x64.
Luckily they can be smaller nowadays. Before I left home my room had more towers of computers than it had furniture. In my first flat I had a full size rack!
pv starfive-jh7110-VF2_515_v2.5.0-69-minimal-desktop.img \
| sudo dd of=/dev/sda
We need to repartition, because the boot partition is way too small. It only fits one kernel/initrd, which became a problem I ran into.
Unfortunately gparted doesn’t seem to work on disk images. It Continue reading
I finally got around to learn Rust. Well, starting to.
It’s amazing.
I’m comparing this to learning Basic, C, C++, Erlang, Fortran, Go, Javascript, Pascal, Perl, PHP, Prolog and Python. I wouldn’t say I know all these languages well, but I do know C, C++, Go, and Python pretty well.
I can critique all these languages, but I’ve not found anything frustrating or stupid in Rust yet.
Rust is like taking C++, but all the tricky parts are now the default, and checked at compile time.
With C++11 we got move semantics, so we have to carefully disable the (usually) default-created copy constructor and copy assignments, or if you have to allow copies then every single use of the type has to be very careful to not trigger copies unless absolutely necessary.
And you have to consider exactly when RVO kicks in. And even
the best of us will sometimes get it wrong, especially with
refactors. E.g. who would have thought that adding this optimization
would suddenly trigger a copy of the potentially very heavy Obj
object, a copy that was not there before?
--- before.cc 2022-12-28 10:32:50.969273274 +0000
+++ after.cc 2022-12-28 10:32:50.969273274 +0000
Continue readingIn my blog post The uselessness of bash I made a tool to improve pipes in shell, to assemble a better pipeline.
It solves the problem, but it’s a bit too different, with its own language.
While complaining with some people at work that one of the main features of shell (the pipe operator) is broken, someone joked that it should be replaced by a protobuf based protocol.
But on second thought it’s not really a joke.
How about instead of this:
$ goodpipe <<EOF
[
["gsutil", "cat", "gs://example/input-unsorted.txt"],
["sort", "-S300M", "-n"],
["gzip", "-9"],
["gsutil", "cp", "-", "gs://example/input-sorted-numerically.txt.gz"]
]
EOF
how about this:
$ wp -o gsutil cat gs://example/input-unsorted.txt \
| wp -io sort -S300M -n \
| wp -io gzip -9 \
| wp -i gsutil cp - gs://example/input-sorted-numerically.txt.gz
It doesn’t use protobufs, but a simpler regular protocol. This in order to avoid well known bugs types. Before implementing any protocol also see formal theory and science of insecurity.
First I hacked it together in Go, but I think the main implementation I’ll maintain is the one I made while porting it to Rust, as a way to learn Rust. The Continue reading
Everyone’s talking about it, so why should I not add something too.
ChatGPT is pretty amazing. I asked it an interview question, and it just plain solved it. Well, there was one real world imperfection. I asked it to fix that problem, and then it went off the rails. The more I hinted, the worse the code got.
I won’t give the details here, because I want to keep using the question / topic.
So I wondered if it could find bugs. First an easy one:
Q:
What's the problem with this C++ code:
int* foo() {
auto ret = new int;
int r2 = *ret * 2;
return &r2;
}
A:
There are several problems with the code you provided:
1. The function foo is returning a pointer to a local variable r2,
which is not safe because the memory for r2 is deallocated when the
function returns. This can cause undefined behavior when you try to
access the returned pointer after the function has returned.
2. The function foo is allocating memory for an integer with new int,
but it is not deallocating that memory before the function
returns. This can lead to a Continue readingAnd now for something completely different.
I’ve off and on thought about this for years, so it needed to be written down.
Obviously SPOILERS, for basically all Terminator movies.
Summary of reasons
I’m aware of The Terminator Wiki, but I don’t care about it. My opinions are on the movies as movies.
In Terminator 1 (T1) Arnold is clearly a robot in human skin. At no point do you believe it’s a human. The only reason people don’t stop and scream and point, is that “I’m being silly, that’s clearly impossible”. But Arnold spends the whole movie in the uncanny valley, the kind in 2022 reserved for realistically generated CGI characters.
It’s very nearly a perfect movie. Just take his first dialog. “Nice night for a walk”, the punks say. They are saying this to a machine that has never talked to a human before, so its response is complete nonsense. It just repeats the words back to them.
It’s a Continue reading
I’m writing a webserver optimized for serving a static site with as high performance as possible. Counting every syscall, and every copy between userspace and kernel space.
It’s called “tarweb”, because it serves a website entirely from a tar file.
I’m optimizing for latency of requests, throughput of the server, and scalability over number of active connections.
I won’t go so far as to implement a user space network driver to bypass the kernel, because I want to be able to just run it in normal setups, even as non-root.
I’m not even close to done, and the code is a mess, but here are some thoughts for now.
Every syscall costs performance, so we want to minimize those.
The minimum set of syscalls for a webserver handling a request is:
accept() to acquire the new connection.epoll_ctl() to add the fd.epoll_wait() & read() or similar. (ideally getting the whole
request in one read() call)epoll_wait() & write() or similar. (again ideally in one call)close() the connection.There’s not much to do about accept() and read(), as far as I can
see. You need to accept the connection, and you need to Continue reading
Floating point can be tricky. You can’t really check for equality, and
with IEEE 754 you have a bunch of fun things like values of not a
number, infinities, and positive and negative zero.
But integers are simple, right? Nope.
I’ll use “integers” to refer to all integer types. E.g. C’s int,
unsigned int, gid_t, size_t, ssize_t, unsigned long long, and Java’s
int, Integer, etc…
Let’s list some problems:
Casting an integer from one type to another changes three things:
The first is obvious, and is even safe for the language to do implicitly. Why even bother telling the human that a conversion was done?
But think about the other two for a minute. Is there any reason that you want your Continue reading
There are a few ways to attempt to parse a string into a number in the C standard library. They are ALL broken.
Leaving aside the wide character versions, and staying with long
(skipping int, long long or intmax_t, these variants all having
the same problem) there are three ways I can think of:
atol()strtol() / strtoul()sscanf()They are all broken.
I’ll start by claiming a common sense “I know it when I see it”. The
number that I see in the string with my eyeballs must be the numerical
value stored in the appropriate data type. “123” must be turned into
the number 123.
Another criteria is that the WHOLE number must be parsed. It is not OK to stop at the first sign of trouble, and return whatever maybe is right. “123timmy” is not a number, nor is the empty string.
Failing to provide the above must be an error. Or at least as the user of the parser I must have the option to know if it happened.
atol()| Input | Output |
|---|---|
| 123timmy | 123 |
| 99999999999999999999999999999999 | LONG_MAX |
| timmy | 0 |
| empty string | 0 |
" " |
0 |
No. All Continue reading
The title of this post is clearly a reference to the classic article PHP a fractal of bad design. I’m not saying Java is as bad as that, but that it has its own problems.
Do note that this post is mostly opinion.
And I’m not saying any language is perfect, so I’m not inviting “but what about C++’s so-and-so?”.
What I mean by “bad experiments” is that I don’t think the decisions the creators of Java were bad with the information they had at the time, but that with the benefit of hindsight they have proven to be ideas and experiments that turned out to be bad.
Ok, one more disclaimer: In some parts here I’m not being precise. I feel like I have to say that I know that, to try to reduce the anger from Java fans being upset about me critiqueing their language.
Don’t identify with a language. You are not your tool.
A lot of Java’s problems come from the fact that it’s too object oriented. It behaves as if everything is axiomatically an object.
No free-standing functions allowed. So code is full of public static
functions, in classes with no Continue reading
I recently got fiber to my house. Yay! So after getting hooked up I started measuring that everything looked sane and performant.
I encountered two issues. Normal people would not notice or be bothered by either of them. But I’m not normal people.
I’m still working on one of the issues (and may not be able to disclose the details anyway, as the root cause may be confidential), so today’s issue is traceroute.
In summary: A bad MPLS config can break traceroute outside of the MPLS network.
$ traceroute -q 1 seattle.gov
traceroute to seattle.gov (156.74.251.21), 30 hops max, 60 byte packets
1 192.168.x.x (192.168.x.x) 0.302 ms <-- my router
2 194.6.x.x.g.network (194.6.x.x) 3.347 ms
3 10.102.3.45 (10.102.3.45) 3.391 ms
4 10.102.2.29 (10.102.2.29) 2.841 ms
5 10.102.2.25 (10.102.2.25) 2.321 ms
6 10.102.1.0 (10.102.1.0) 3.454 ms
7 10.200.200.4 (10.200.200.4) 2. Continue readingIf you’re writing a tool that takes untrusted input, and you should treat almost all input as untrusted, then it’s a good idea to add a layer of defense against bugs in your code.
What good is a buffer overflow, if the process is fully sandboxed?
This applies to both processes running as root, and as normal users. Though there are some differences.
In POSIX you can only sandbox if you are root. The filesystem can be
hidden with chroot(), and you can then change user to be non-root
using setuid() and setgid().
There have been ways to break out of a chroot() jail, but if you
make sure to drop root privileges then chroot() is pretty effective
at preventing opening new files and running any new programs.
But which directory? Ideally you want it to be:
The best way no ensure this is probably to create a temporary directory yourself, owned by root.
This is pretty tricky to do, though:
// Return 0 on success.
int do_chroot()
{
const char* tmpdir = getenv("TMPDIR");
if (tmpdir == NULL) Continue readingI’ll just assert that there’s no way to use seccomp()
correctly. Just like how there’s no way to use gets()
correctly, causing it to eventually be removed from the C and C++
standards.
seccomp allows you to filter syscalls with a ruleset.
The obvious thing is to filter anything your program isn’t supposed to be doing. If it doesn’t do file IO, don’t let it open files. If it’s not supposed to execute anything, don’t let it do that.
But whether you use a whitelist (e.g. only allow working with already open file descriptors), or a blacklist (e.g. don’t allow it to open these files), it’s fundamentally flawed.
open() in your code actually becomes the openat syscall. Maybe. At
least today. At least on my machine, today.
select() actually becomes pselect6. At least on Fridays.
If you upgrade libc or distribute a binary to other systems, this may start to fail.
Calling printf() will call the syscall newfstatat, a syscall hard
to even parse into words. But only the first time you call it! So
after your first printf() you can block newfstatat.
Maybe Continue reading
Setting up AX.25 over 1200bps was easy enough. For 9600 I got kernel panics on the raspberry pi, so I wrote my own AX.25 stack.
But I also want to try to run AX.25 over D-Star. Why? Because then I can use radios not capable of 9600 AX.25, and because it’s fun.
It seems that radios (at least the two I’ve been working with) expose the D-Star data channel as a byte stream coming over a serial connection. Unlike working with a TNC you don’t have to talk KISS to turn the byte stream into packets, and vice versa.
The first hurdle to overcome, because we want to send binary data, is to escape the XON/XOFF flow control characters that the IC9700 mandates. Otherwise we won’t be able to send 0x13 or 0x11. Other bytes seem to go through just fine.
So I wrote a wrapper for that, taking /dev/ttyUSB1 on one
side, and turning it into (e.g.) /dev/pts/20 for use with
kissattach.
$ ./dsax /dev/ttyUSB1
/dev/pts/20
$ kissattach /dev/pts/20 radio
$ kissattach -p radio -c 2 # See below
Set Menu>Set>DV/DD Set>DV Data TX to Auto, for “automatic PTT”. As
Continue reading
If you only have your app in English then you’ll still be understood[1] by the new market whose official language isn’t English.
If you show farenheit (a word I can’t even spell), then 96% of the world cannot understand your app. At all.
For most of the west I would argue that translation doesn’t even matter at all, but you cannot have your app start your weeks on Sunday, you cannot show fahrenheit, or feet, or furlongs, or cubits or whatever US-only units exist. And you cannot use MM/DD/YY.
NONE of these things are tied to language. Most users of English don’t want any of this US-only failure to communicate.
[1] While most of the world doesn’t speak English fluently, they may know words. And they can look up words. You cannot “look up” understanding fahrenheit or US-only date formats.
A while ago I was asked why I wrote Sim in C++ instead of Go. I stumbled upon my answer again and realized it could be a blog post.
So here’s what I wrote then. I think I stand by it still, and I don’t think the situation has improved.
My previous experience with “low level” things in Go (being very
careful about which syscalls are used, and in which order) has had
some frustrations in Go. Especially with portability. E.g. different
definitions of syscall.Select between BSDs and Linux, making me have
to use reflection at some points. (e.g. see
this Go bug.
And to work around those things Go unfortunately uses the antipattern
of (essentially) #ifdef __OpenBSD__, which we’ve known for decades
is vastly inferior to checking for specific capabilities.
To me the Go proverb “Syscall must always be guarded with build
tags” essentially means “Go is not an option for any
program that needs to be portable and may at some point in the future
require the syscalls package”. And since this tool is meant to be
portable, and calls what would be syscall.Setresuid, Continue reading
In my previous two posts I set up a login prompt on a bluetooth serial port and then switched to running SSH on it.
I explicitly did not set up an IP network over bluetooth as I want to minimize the number of configurations (e.g. IP address) and increase the chance of it working when needed.
E.g. firewall misconfiguration or Linux’s various “clever” network managers that tend to wipe out network interface configs would have more of a shared fate with the primary access method (SSH over normal network).
This post is about how to accomplish this more properly.
The problems now being solved are:
It wasn’t entirely reliable. The rfcomm tool is pretty buggy.
There was no authentication of the Bluetooth channel. Not as much a problem when doing SSH, but if there are passwords then there could be a man-in-the-middle attack.
The server side had to remain discoverable forever. So anyone who scans for nearby bluetooth devices would see your servers, and would be able to connect, possibly brute forcing passwords. Not as much of a problem if running SSH with password authentication turned off, but why broadcast the name of a server if you don’t Continue reading
Yesterday I set up a simple serial console over bluetooth as a backup console.
Today I’m running SSH over bluetooth. Raw SSH, no IP. I only use IP on the two ends to talk to the SSH client and server. It doesn’t actually go over the bluetooth.
This fixes the security aspects with the previous solution. As long as you make sure to check the host key signature it’ll be perfectly secure.
No need for one-time passwords. You can even use SSH pubkey auth.
Server:
rfcomm watch hci0 2 socat TCP:127.0.0.1:22 file:/proc/self/fd/6,b115200,raw,echo=0
Client:
sudo rfcomm bind rfcomm2 AA:BB:CC:XX:YY:ZZ 2
ssh -oProxyCommand="socat - file:/dev/rfcomm2,b115200,raw,echo=0" dummy-hostname
If you’re messing around with an OpenSSH config then it may be a good idea to set up a minimal config on another port. Maybe port 23. Not like that port is used for anything else anymore.
Sometimes you want to connect to a bluetooth on the console. Likely because you screwed something up with the network or filewall settings.
You could plug in a screen and keyboard, but that’s a hassle. And maybe you didn’t prepare the Pi to force the monitor to be on even if it’s not connected at boot. Then it just doesn’t work.
Even more of a hassle is to plug in a serial console cable into the GPIO pins.
But modern Raspberry Pi’s have bluetooth. So let’s use that!
Create /etc/systemd/system/bluetooth-console.service with this content:
[Unit]
Description=Bluetooth console
After=bluetooth.service
Requires=bluetooth.service
[Service]
ExecStart=/usr/bin/rfcomm watch hci0 1 getty rfcomm0 115200 vt100
Restart=always
RestartSec=10
StartLimitIntervalSec=0
[Install]
WantedBy=multi-user.target
This sets up a console on bluetooth channel 1 with a login prompt. But
it doesn’t work yet. Apparently setting After, Required, and even
Requisite doesn’t prevent systemd from running this before setting
up bluetooth (timestamps in the logs don’t lie). Hence the restart stuff.
I also tried setting ExecStartPre / ExecStartPost there to enable
Bluetooth discoverability, since something else in the boot process
seems to turn it back off if I set it Continue reading