Question about system logs

[felix91gr]

Hi Tony and others (if there are) ^^

I'm helping at the langserver-swift project and we'd like to have some cross-platform logging. And since I know for a fact that TraceLog does work on both Darwin and Linux, I'm thinking it might be the perfect tool for the job.

Yesterday when we were talking about this, Ryan said that we should be outputting the log messages to Unified logging on Darwin and to the systemd logging journal on Linux. I think that's reasonable. Is is possible to configure TraceLog to do that? I know close to nothing about system logs so please forgive me if I'm noobing too much here. I know I probably am 😅

[tonystone]

Hi @felix91gr, langserver-swift looks like a very cool project.

So on logging and TraceLog, yes, TraceLog was designed for use-cases like this, it allows multiple pluggable Writers that write the low-level information to various logging systems, databases, sockets, or whatever you'd like. You can install multiple so it can write the say the console and a file log at the same time. If you install your own writer, this removes the default writer.

The writer interface is straightforward, you basically just have to write the init code (if there is any ) and implement this method.

func log(_ timestamp: Double, level: LogLevel, tag: String, message: String, runtimeContext: RuntimeContext, staticContext: StaticContext)

For os_log you would use the os_log(_:dso:log:type:_:) method directly. The minor complication I see with this (and this would be with all loggers, not just TraceLog) is the #dsohandle, this is an opaque pointer that Apple uses to get the file, line, and func information for the calling context (TraceLog currently uses #file, #func, #line for capturing that). This would need to be passed in from the origin. TraceLog could easily be modified in a non-breaking fashion to capture this. I could do that for you.

The Writer for os_log would be straightforward, really, aside from the method and class declaration itself, just a one-liner. It also would be a worthy addition to TraceLog.

On the systemd journal. This should also be easy also I think there would be a little more to the writer since you would have to set up the address and ports to log to in the init method of your Writer, in the end, the write method would be very simple again. I believe in the shell you would use systemd-cat to log messages, not sure if that or another method is exposed in swift yet, would have to look into this. If it's not it would be a simple matter of creating a modulemap file to expose the C interface to Swift.

Both seem like worthy additions to TraceLog.

---

Update: I took a few moments to write an os_log writer (OSLogWriter) for TraceLog.

Take a look at branch os-log-writer.

Still need to figure out how to write tests for it and also may need a little cleanup.

[felix91gr]

This would need to be passed in from the origin. TraceLog could easily be modified in a non-breaking fashion to capture this. I could do that for you.

The Writer for os_log would be straightforward, really, aside from the method and class declaration itself, just a one-liner. It also would be a worthy addition to TraceLog.

Oh my gosh, that would be awesome! Please do 🙏

On the systemd journal. This should also be easy also I think there would be a little more to the writer since you would have to set up the address and ports to log to in the init method of your Writer, in the end, the write method would be very simple again. I believe in the shell you would use systemd-cat to log messages, not sure if that or another method is exposed in swift yet, would have to look into this. If it's not it would be a simple matter of creating a modulemap file to expose the C interface to Swift.

Both seem like worthy additions to TraceLog.

I'll look it up too. Let's see what we can find. If you need one, I could lend you a hand for testing this, since I have my Swift sat up on Linux :)

Again. thank you so much for your help 🙇‍♂️

[tonystone]

@felix91gr, would you want to contribute to the systemd Writer? We could work on it together. You have your environment setup, and you are familiar with your particular requirements, so that is a huge plus.

[felix91gr]

Sure 👍 I'd love to

[felix91gr]

Useful info for this:

Here's the documentation for logging into systemd.

I'm almost certain that we can't import it directly from Swift like we can import GLibc, but I need to read more on it to be sure. If not, we should make the modulemap.

Edit:

In the future, we might not even need modulemaps, so that'll be a cool day ^^. I don't think that's included in Swift 4.1, but it should be coming up for 4.2 or 5.0 :)

---

PS: I've had success in writing a logger that outputs logs into a file. Now that I understand that, I see why you chose such a design. Adding the canonical system logs would only require adding like 50 lines or less of Swift code in a new file :)

[tonystone]

That does indeed look like the correct calls. In addition, I found this helpful guide "systemd for Developers III", by Pid Eins.

Note that syslog can also be used (syslog is included in GLibc on Linux, see the modulemap for Glibc). The biggest issue I see with this is we would be missing the code file, func, line information which I believe is a show stopper for syslog.

From the sd_journal.h file I believe that the prefered calls for us would be:

   int sd_journal_print_with_location(int priority, const char *file, const char *line, const char *func, const char *format, ...)

Since this call contains the priority, file, line, and func which we would like to maintain.

---

P.S. TraceLog can certainly use a file writer so if you want to make your file work generic and contribute that, it would be a great addition. I could see it requiring the ability to choose a file location and name as well as file rotation and possibly some kind of max space it can take up before it starts to delete files.

[RLovelett]

I think that syslog is a variadic function. And last time I checked variadic functions are not supported by Swift without some sort of wrapper.

[felix91gr]

And last time I checked variadic functions are not supported by Swift without some sort of wrapper.

Ah, dammit. That may be why my attempts at making a Swift wrapper aren't working.

We'll need a C wrapper, then. That's not so hard to provide :)
Gimme 10 minutes, I think I know what my code is missing now.

Edit: this is what I'm working on, btw

[felix91gr]

Okay, it's ready. But it still fails. I think it may be because I need to specify a pkgConfig command such that SPM can find the C header in my system.

But for using pkgConfig, the wrapper has to be a System Module package... Anyone here knows how to make one of those? I've tried the official documentation but the exact code posted there doesn't work 🤔

---

Btw, for accessing systemd on Debian-based Linuxes, this might be necessary:

sudo apt-get install libsystemd-dev

[felix91gr]

I now just realized that you were talking about syslog. We're trying to work with sd_journal_print, which seems to be the modern replacement for syslog. But nevertheless the problem you pointed at (variadic functions not being accessible) is there. The sd_journal_print function is variadic as well.

[felix91gr]

And, last answer because I feel like I'm spamming y'all:

@tonystone,

From the sd_journal.h file I believe that the prefered calls for us would be:

int sd_journal_print_with_location(int priority, const char *file, const char *line, const char *func, const char *format, ...)

Since this call contains the priority, file, line, and func which we would like to maintain.

I'd tend to agree, but I don't think that's the intended usage of that function. If you take a look at the headers, it says: "Used by the macros below. You probably don't want to call this directly.". Can we make-do with sd_journal_print? Since it's part of the official API, we would be better off if we could use it instead.

P.S. TraceLog can certainly use a file writer so if you want to make your file work generic and contribute that, it would be a great addition.

Thanks for the offer! I'd like to do it, but maybe not right now. I have winter holidays in July, and with the break from uni I should be able to dedicate myself to that feature mindfully :)

Otherwise, please feel free to take the current implementation at traceLog-usage and make it complete with things like what you mentioned:

I could see it requiring the ability to choose a file location and name as well as file rotation and possibly some kind of max space it can take up before it starts to delete files.

To which btw, I agree :)