I was looking at some GitHub actions and noticed a literal, pipe |, with a dash, -, and had no idea what it meant. After a bit of searching, I found YAML Multiline which showed the effects of the feature when making multi-line strings. I.e. run: |- vs run: |
The dash is the block chomping indicator which lets yaml know if it should keep the newline at the end of the multi-line string. If you chomp it strips the new line. If you just use the pipe, it’ll contain the last new line.
Now does it really matter in GitHub actions? I don’t really think so. Most shells will strip the last newline character of a command.
So there you go. Pipe dash strips the last end line character of the multiline lateral command.
Howdy Everyone. I trained machines to predict Team Fortress 2, TF2, spell prices. I published the code at my joekiller spells repo. Don’t forget, you can get Splack, spell listings crack, at https://discord.gg/BSSrHRd.
My discord has nearly every spell listing since April 2022. It predates the backpack.tf websocket service and is sophisticated like your mother.
This article outlines the steps taken to sniff and review SSL traffic of a NodeJS application I was working with. It covers setting up the intercept using Charles Proxy. Links for alternative means are included as well. The approach is similar for any proxy tool. The reason I outlined this is to give an overview for TF2Autobot specifically and to remind others that Node doesn’t support proxy out of the box for its native http and https calls using environment variables. I was searching for NODE_DEBUG=’https’ commands and came to the conclusion that just using a Debug Proxy (Charles Proxy or Proxyman) is the easiest.
To setup the proxy, I used a tip I learned when I was inflicted with Netskope and they have a nice article about how to how to make development tools work with SSL interception. They have instructions for integration MITM network security with Node.JS and other tools like AWS CLI. NodeJS they mention the following which is helpful because this is how you can easily add a Debug Proxy to your NodeJS process. In my case, I’m debugging TF2Autobot trying to get an understanding of how it is working with the backpack.tf public API. Netskope explains for NodeJS the best approach is to use the Combined Certificate bundle at let Node know about it “NODE_EXTRA_CA_CERTS” environment variable. Here is Netskope’s explanation.
export NODE_EXTRA_CA_CERTS=[your CA certificate file path]
The main difference between NODE_EXTRA_CA_CERTS and the cafile config property is that the former adds a cert, whereas the cafile config property replaces the certs. For those that just want to add a corporate cert to the chain, NODE_EXTRA_CA_CERTS is the easier option
To get the https traffic off my application I need to get the Charles Proxy SSL certificate. The details are explained in their SSL Proxying Documentation. I downloaded the “base 64 pem” by clicking Help then navigating to get the certificate via: Help > SSL Proxying > Save Charles Root Certificate…
The “Save Charles Root Certificate” dialog wants a directory selected. Typically it’ll open the Save chooser in your home directory. Don’t bother trying to put it in another directory because the save dialog didn’t respect my ask and just saved it under the first spot it suggests it’ll save it. My certificate was save under my username path, i.e. "C:\Users\username\charlesProxy.pem".
Disable some of Charles Proxy Windows sniffing
Charles is very good out of the box for windows. I found that I actually wanted to disabled the default Windows proxy. To do so I used the “Proxy” menu and then selected “Proxy Settings”. Under the Windows tab of the Proxy Settings dialog, disable “Enable Windows proxy” and “Enable Windows proxy on launch”. Since Windows Proxy was already enabled. Once I turned it off I had to restart the program to make it stop sniffing everything else. I tried to see if it would pick up the NodeJS traffic and it wouldn’t just out of the box. So disabling this feature allows the traffic to just be on the program I’m working on.
To make sure the traffic was captured, I added a wildcard match for Include Locations under the SSL Proxying Settings.
Configure TF2Autobot to use Charles Proxy
There are two items to take care of here, configuring NodeJS to enabled using extra CA Certificates and then adding HTTPS_PROXY to enable axios to pipe traffic through Charles Proxy. The extra CA certificates tells NodeJS to trust certs that Charles Proxy will create to man in the middle (MITM) the SSL traffic that is being proxied.
Once I had the certificate I was ready to configure my bot environment so that I may debug the traffic. I cannot put it in the .env of the bot as that only injects env into the program, not NodeJS itself. I think the ecosystem.json files are fine but I’m just running the following for my config.
Because I’m just running this as a development exercise in Webstorm, I’m going to set the Environment variables of my Run Configuration. VSCode has instructions that include how to modify a launch.json to include env variables.
Running and Intercepting the Application Traffic
Okay so almost ready. First I check to make sure Charles is ready by ensuring the red record button is enabled, the pad lock icon is enabled (SSL intercept), and I click the broom icon to clean out any old session stuff.
With the proxy good, I prepare the code with npm install and npm run build. Then I start the bot with the configuration that includes the CA bundle variable and proxy settings and traffic can be reviewed.
Netskope has a script which they lock behind a support login which I believe does SDK and CLI specific CA certificate bundles like in the CLI specific option of ca_bundle however I found, as mentioned in the docs, that just setting the AWS_CA_BUNDLE environment variable is much more efficient to get it to work for Amplify CLI, CDK, and any derived tools that use AWS SDK. Make sure you point it at the “Combined Certificate Bundle”.
Joe's WebSocket ManagerA simple interface to connect to a WebSocket and receive an event
when messages are pending.
The WebSocket manager has been used for some time in a side project and is very solid. The Socket To Interface Utility was a quick project I did yesterday to capture a single interface over a disparity of messages that start similar at the root but towards the leaves are very different.
If MariusAlch were on twitter, I’d @ him and say, “Thank you so much for your useful json-to-ts utility!”
Json to TS converts JSON to Typescript interfaces. It’s especially useful because it converts arrays of different sets to a unified interface. I used it to catch messages of varying payloads from Backpack.tf’s websocket and create a nice common interface for the messages. The payload are mainly homogeneous near the root of the object but it’s somewhat scattered towards the leaves. You can see from this example backpack.tf websocket payload how dense the data can be.
My utility does a deep examination of any messages’s structure and simplifies it to an abbreviated version which I called the “Shape” of the object.
type Shape = {
[key: string]: string | null | boolean |
number | Shape[]
} | string | null | boolean | number
You can see how the type is concise and sufficient for generic JSON message Interface detection. A sample of the tests are as follows.
While collecting messages, I take a hash of the Shape and, if I get a new sample, I add it to my collection. Once I’ve taken a sufficient sampling, the software assembles all the messages into an object that has my root type as the key and a vector to hold the samples. I pass the object to JsonToTs() and, “huzzah!”, out pops all my interfaces with optional types nicely intertwined.
When trying to be diligent about keeping strict types via ESLint in a TypeScript based ExpressJS, You may hit the dreaded, “ESLint: Unsafe call of an any typed value. (@typescript-eslint/no-unsafe-call)” while trying to use the default express() method via:
import express from 'express';
const app = express();
After an npm i -D @types/express the error will remain. After searching around and seeing so many fixes for importing Request and Response types, I finally found an explanation of how to fix the any typed value error on the express() method via smichel17’s response to, “express() function and express namespace ambiguity #37920”. The nuance is in that “import” returns plain objects and “require” returns returns objects and functions. Note that you cannot destructure the “require” call so to get your “Request” and “Response” types without using “express.Request” and “express.Response” you still need the traditional import statement as well.
Update your code to the following and resolve the “Unsafe call of an any typed value” error without using the dreaded escape hatch of "noImplicitAny": false.
import express = require('express');
import { Request, Response } from 'express';
const app = express();
const port = 3000;
app.get('/', (req: Request, res: Response) => {
res.status(200).send(
'Express + TypeScript Server ' +
'With Strict Types by Joe');
});
app.listen(port, () => {
console.log(
`⚡️[server]: Server is running ' +
' at http://localhost:${port}`);
});
joekiller 