All posts by joekiller

I solve hard problems and have fun doing it.

aws-cdk template porting migration tips

AWS-CDK Migration Tips

When migrating to a new framework there are going to be some working pains. This is just a collection of frustrations that I encountered while adopting aws-cdk. I did some research prior and found the article “Hey CDK, how can I migrate my existing CloudFormation templates?” by Philipp Garbe and the “core module AWS CDK” documentation most helpful in thinking about migrating initially.

Import Immutable Roles

Use the mutable flag when importing existing roles with Role.fromRoleArn() otherwise the precision of the aws-cdk may lead to the dreaded “Maximum policy size of 10240 bytes” error. Eventually aws-cdk issue #4465 will be fixed and we will welcome the precise IAM policies the CDK generates.

The maximum policy size error was most often encountered on CodePipeline deploy roles where we had a large number of independent artifacts deploying CloudFormations.

Explicit to_string() in python

Having to explicitly call the core.Fn.get_att(‘Foo’, ‘Bar’).to_string() operator instead of using a str() for f'{var}’ style tripped me up.

I noticed in my IDE that the signature called for a string (thanks types!) so I tried:

str(core.Fn.get_att('Foo','Bar'))

and

f'{core.Fn.get_att('Foo','Bar')}'

but because only __repr__ is defined in the python interface I got an ugly object name when I expected __str__ to be implemented. I overlooked the to_string(), which is a pretty common method for many object oriented languages, as I expected the class to behave more pythonically.

Beware of Copy Paste / Naming

Name a stack the same as another? You get a diff but if you aren’t paying attention you’ll blow away a stack before realizing it. Also you end up with the old stack as well not updated because of this config SNAFU.

Import Pains

CfnImport is great for importing old CloudFormations but the stack is immutable upon import. Any changes to the stack must happen prior to making the call. We leaned on PyYAML but then had to undo a few of the niceties of only processing the template with AWS based systems.

Intrinsic Function Shortcuts

For example all bang, “!” ie “!Ref” or “!Sub”, references need to be updated to be the full function command, ie “Ref:” and “Fn::Sub:”.

Attributeerror: ‘datetime.date’

IAM Policy Documents specifying the Version unquoted instead of as a string, ie Version: 2012-10-17 instead of Version: ‘2012-10-17’, will have the cdk synth command greet them with following obscure error.

This error also occurs on AWSTemplateFormatVersion blocks so beware.

 AttributeError: 'datetime.date' object has no attribute '__jsii__type__'.

Example CfnImport

Here is an example of using the CfnImport to inject parameters into a traditional template and then load it into the CDK stack.

raw_stack.py gist link

 

Fixing unhandled instruction bytes error Running Valgrind on AWS CodeBuild

When running Valgrind against one of our C libraries we encountered some discrepancies in the build where locally all would pass but on AWS CodeBuild using the aws/codebuild/standard:2.0 image we would get errors like:

vex amd64->IR: unhandled instruction bytes: 0x62 0xF1 0x7D 0x48 0xEF 0xC0 0xC5 0xF9 0x2E 0x45

The full message was like:

==16128== Memcheck, a memory error detector
==16128== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
==16128== Using Valgrind-3.13.0 and LibVEX; rerun with -h for copyright info
==16128== Command: /root/build/meh/test/.libs/state
==16128== 
[==========] Running 2 test(s).
[ RUN      ] test1
vex amd64->IR: unhandled instruction bytes: 0x62 0xF1 0x7D 0x48 0xEF 0xC0 0xC5 0xF9 0x2E 0x45
vex amd64->IR:   REX=0 REX.W=0 REX.R=0 REX.X=0 REX.B=0
vex amd64->IR:   VEX=0 VEX.L=0 VEX.nVVVV=0x0 ESC=NONE
vex amd64->IR:   PFX.66=0 PFX.F2=0 PFX.F3=0
==16128== valgrind: Unrecognised instruction at address 0x1173bd.
=...
==16128== Your program just tried to execute an instruction that Valgrind
==16128== did not recognise.  There are two possible reasons for this.
==16128== 1. Your program has a bug and erroneously jumped to a non-code
==16128==    location.  If you are running Memcheck and you just saw a
==16128==    warning about a bad jump, it's probably your program's fault.
==16128== 2. The instruction is legitimate but Valgrind doesn't handle it,
==16128==    i.e. it's Valgrind's fault.  If you think this is the case or
==16128==    you are not sure, please let us know and we'll try to fix it.
==16128== Either way, Valgrind will now raise a SIGILL signal which will
==16128== probably kill your program.

The error seems to indicate that the architecture doesn’t seem to match what the docker image had so going off of a Linux Headers Reinstall article we added the following and then the architecture packages were fine.

apt upgrade --fix-missing -y && apt autoremove -y && apt autoclean -y

 

AWS CDK CLI can only be used with apps created by CDK error

I upgraded my AWS CDK to 1.10.1 today because it prompted me via:

**************************************************
*** Newer version of CDK is available [1.10.0] ***
*** Upgrade recommended                        ***
**************************************************

After doing the upgrade via

npm install -i -g aws-cdk

I went to do a cdk ls or cdk diff and was greeted with the error:

CDK CLI can only be used with apps created by CDK >= 1.10.0

Googling around wasn’t too helpful but finally I figured out that it was complaining that my python dependencies  had the old aws-cdk libraries installed.

A quick

rm -r .env/
python -m venv .env
pip install -r requirements.txt

And I was back in business

cdk ls
integration-pipeline

ZyXEL EMG3425-Q10A Port Forwarding

The ZyXEL EMG3425-Q10A has NAT Port Forwaring but it doesn’t seem to work well because the Remote Management section has been patched out. This causes the remote management screen to always boot on the IP that is the same as the Port Forwarding Default Server Setup. You need to do two things to fix this mess. First change the default server setup to be the target you normally will port forward to. Turn on a firewall on this host. Second, you need to forward the WWW and HTTPS rules to that host.

When adding additional rules, click “add” instead of apply. Add ports like 25565 to host minecraft or 27015 to TF2.

You cannot delete rules or you have to re-enter all of them in the right order. AGAIN.

EDID Reading on Arch Linux

There are several tools to read the Extended Display Identification Data, EDID, from systems but I found LinuxTV’s edid-decode the most thorough when debugging for a linux 5.0.x display boot flicking problem.

On arch I ran installed edid-decode-git and then ran a quick script:

for f in `find /sys/devices -name 'edid'`; do sudo cat $f| edid-decode;done

and I got something like:

EDID version: 1.4
Manufacturer: BOE Model 65a Serial Number 0
Made in week 1 of 2015
Digital display
6 bits per primary color channel
DisplayPort interface
Maximum image size: 34 cm x 19 cm
Gamma: 2.20
Supported color formats: RGB 4:4:4, YCrCb 4:4:4
First detailed timing includes the native pixel format and preferred refresh rate
Display x,y Chromaticity:
  Red:   0.6416, 0.3437
  Green: 0.3183, 0.6103
  Blue:  0.1494, 0.0439
  White: 0.3125, 0.3281
Established timings supported:
Standard timings supported:
Detailed mode: Clock 139.770 MHz, 344 mm x 194 mm
               1920 1968 2000 2080 hborder 0
               1080 1083 1089 1120 vborder 0
               +hsync -vsync 
               VertFreq: 59 Hz, HorFreq: 67197 Hz
Detailed mode: Clock 111.820 MHz, 344 mm x 194 mm
               1920 1968 2000 2080 hborder 0
               1080 1083 1089 1120 vborder 0
               +hsync -vsync 
               VertFreq: 47 Hz, HorFreq: 53759 Hz
ASCII string: J125V
Manufacturer-specified data, tag 0
Checksum: 0xa9 (valid)

This helped when trying to diagnose: black screen on Dell XPS 15 with kernel 5.0 and Bug 109959 – REGRESSION: black screen with linux 5.0 when starting X

Socialism?

Did you know that employee owned companies are practicing socialism? Anything the government does, building parks, roads, defense spending, etc is technically supposed to be socialism. Ie, the government isn’t there to make a profit. We the people elect representatives to decide where our extra money goes. Socialism is pooling profits and redistributing them as the people see fit vs as the boss sees fit.

Socialism put a man in the moon.

Ideal Communism = Government runs everything
Ideal Capitalism = Everyone give profits to companies after being paid
Ideal Socialism = Everyone gets an even share of all profits after being paid

USA is totally a mix of socialism and capitalism. Communist we are not.

I think there are many shades in between where we are, where we’ve been,  and where we should go.

Fixing a Kenmore 71033 Freezer Leak Ice Over

The internet is a great place for the do it yourselfer these days. I have a Kenmore 71033 Elite Refrigerator, 795.71033.010 exactly,  and the freezer had been filling up with ice in the bottom. After a while the ice migrated to leaking as water onto the floor. The kids were pleased with the freezer sized ice cubes they could smash outside however we were tired of it.

Fixing the problem was solved through some googling and I wanted to share a trio of videos that helped work through the problem.

First up was a search on searspartsdirect.com with my model information, 79571033010. The site was helpful in that it had some Question and Answers and the first question was titled, “Leaking from Freezer Area and Ice Buildup”. This was pretty much dead on with my problem. Unfortunately the helpful answer said, “Here is an image of a repair procedure that shows how to remote that back panel inside the freezer compartment: Kenmore Refrigerator Evaporator Grill Removal” followed by a dead link. Regardless of this, it was a great sign that my problem was pretty solved so I search for a repair video and found a general disassembly video that was was easy to follow by RepairClinic.com named, “LG Refrigerator Disassembly“. Watching this while waiting for my Thai food made me confident in my endeavor and I searched on. A little later I found a video with the problem explanation and a good overview of how to fix the freezer water leak: sgrddy’s “Fixing Freezer Water Leak on Kenmore Elite Bottom-Freezer“. He took the time to share experience and his tips such as using hot water and siphon were much appreciated.

Of course here comes the gotcha. The problem with my specific Kenmore Elite was the Evaporator Cover was different than all those I had seen. It had no obvious screws and no obvious prying points.

Being that my refrigerator was sitting in the kitchen powered off and half way disassembled one could appreciate that I really didn’t want to fuck this thing up but also did want to finish the job. I desperately poured over my previous clues. I tried using the internet archive to resurrect that “Kenmore Refrigerator Evaporator Grill Removal” image but with no luck. Nothing comes up for “s7.postimg.org/719voyior/KMrefrig_GRILLremoval1.jpg” but I did now know a new term for the back panel. I googled, “how to remove freezer Refrigerator Evaporator Cover kenmore elite” and hit the jackpot. A video by grateful patron, “Kenmore Elite Refrigerator Freezer Back Panel Removal” answered my confusion of how to remove the freezer back panel, or Refrigerator Evaporator Cover as they call it on the searspartsdirect site. Apparently you need a pull and pry bar which of course I didn’t have in the tool box. Instead, I took one of those Ikea wrenches that of course I kept and of course really was never to be used again but had it. I took it and hammered a 90 degree angle on the end and that was good enough to pry and pull on the upper left side of panel to dislodge the cover. I finished up by melting that ice and problem sovled!

So thank you RepairClinic.com, sgrddy, and grateful patron for your internet contributions. I hope this post ties it together and helps another one day.

and I just hope that freezer doesn’t hit another ice age…

Fin?

 

Linux Client VPN using Meraki Cloud Controller authentication

If you want to VPN into your network using the Meraki Cloud Controller the Client VPN Instructions indicate that you may be out of luck when trying to use xl2tp.

Note: The xl2tp package does not send user credentials properly to the MX when using Meraki Cloud Controller authentication, and this causes the authentication request to fail. Active Directory or RADIUS authentication can be used instead for successful authentication.
Note: The xl2tp package does not send user credentials properly to the MX when using Meraki Cloud Controller authentication, and this causes the authentication request to fail. Active Directory or RADIUS authentication can be used instead for successful authentication.

It turns out that if you setup the IPSEC phase1 and phase2 algorithms then it’ll work.

It took some googling to bring it all around but combined with the Project network-manager-l2tp Github issue 34 of  “IPSec options hard coded” and the Ubuntu question “L2tp IPSEC PSK VPN client on (x)ubuntu 16.04“, I found that setting IPSEC Phase1 Algorithms to 3des-sha1-modp1024 and Phase2 Algorithms to 3des-sha1 works.

Phase1 Algorithms: 3des-sha1-modp1024 Phase2 Algorithms: 3des-sha1
Phase1 Algorithms: 3des-sha1-modp1024 Phase2 Algorithms: 3des-sha1

Now I can connect to the VPN no problem. On Arch Linux!

Participating in Season 3 of Ready Steady Pan

I participated in a competitive Team Fortress 2 tournament, aka an e-sports competitive video games tournament. My journey took place over eight weeks with a scramble weekend to sign up plus seven weeks of the regular season. We did not make the post season. It was a journey in pan play beyond even what the great degroot_keep offers. With over 10,000 kills on my Golden Frying Pan this tournament meant I got to test my mantle of pan with champions.

The journey nearly was halted before I could even begin. My awareness of this tournament was brought forth around the 20th of October when an update came through saying, “Added Ready Stead Pan Season 3 tournament medals”.

TF2 Update Released

4

I was intrigued and googled the first image I saw was:

capture2.jpg

Oh no! Panicked, I read more and instantly was relieved!

5.JPG

After some hesitation of entering such a tournament I jumped into the forums and laid it all out to find a team. With my Golden Frying Pan there was no way I was missing out. The tournament was calling.

6.JPG

7.jpg

With just over 24 hours until the deadline and no one replying to my post, I hit the Discord chat and found my team. We were formed in the last seconds. A ragtag bunch looking for glory in the Fortress world.

The team was named “we exist guys“, team number 648. Literally the last team made. We existed only in the last moments before sign ups finished. Our team was completely formed in the extension of the signups.

9.JPG

Only time would tell how my newly found comrades and I would fare. Our roster was a colorful bunch: Team Captains: Diamond and “Carl, Good to see you” followed by spacy, Axie, Dell Conaghr RSP???, red box, joekiller (myself), – VH – SnakeFawdz, and Burnt Venom.

The first week was tense and fun. King of the Hill was the game mode. Matches were played until a side reached three wins. To win a team needed to hold the capture point, the hill, for three minutes. Lacking any offensive projectile weapons, the game played out curiously compared to other first person shooter environments. With five player classes allowed there was a variance on speed and health. Players took anywhere from three to six hits to take down and all combat was melee frying pans only. There were no lucky critical hits; only loud frying pan clanging engagements. Here you could stare down an opponent for seconds beckoning each other in awkward strafing dances trying to just bring the other’s hit box within one’s own range. Overwhelming numbers were the name of the game. With a frenzied start the game quickly gave way to one side but with health slow to get to people had to leave the point leaving opportunity for lesser numbered engagements, 1 on 1, 2 on 2, etc.

Capture

In the end our team was outmatched once however all the others, despite the final scores, were closely contested. We played 5 of the 7 matches ending up with a record of 3 – 4, 2 – 3 in matches plus one win and loss from forfeit, but it felt closer to 5-2. We vanquished what turned out to be the bottom two teams and were squashed by The Knights of Nye in a shocking 6 minutes and 6 seconds. Other matches took between 18 to 30 minutes. We held our own against the mid tier teams, even the Knights of Nye whom made the playoffs. The team improved over time and had fun.

RED_Tournament_Medal_-_Ready_Steady_Pan

But of course this was journey for championing the Golden Pan. How was it? For three of five matches I was in the top. Ending the season I had 155 kills, 43 assists,106 deaths, and put out a total of 35,189 damage (about 281 scouts). The road was hard but the Golden Pan usually rang more than any other each match. With a solid 2.06 Kill and Assists per death ratio the pan performed well. It was a great journey and in the end everyone will get a Tournament Medal. GG’s everyone.

Stats:

Kills Assists Deaths Damage Damage/Minute Kill and Assists / Death Kills / Death Damage Taken Damage taken / Minute HP* Capture Point Captures

Week 1 20 10 25 6191 287 1.2 0.8 6061 281 28 3
Week 3 36 10 10 7227 385 4.6 3.6 4985 266 55 4
Week 4 45 13 40 9649 364 1.4 1.1 9084 342 63 9
Week 5 2 2 8 1170 190 0.5 0.3 1952 317 7 0
Week 7 52 8 23 10952 371 2.6 2.3 8169 277 76 9
Totals or Average 155 43 106 35189 30251 229 25
Averages 31 8.6 21.2 7037.8 319.4 2.06 1.62 6050.2 296.6 5

HP is “Health pickup rating: Small 1p, Medium 2p, Large 4p”

Extra:

Week 1

https://pan.tf/matches/209
http://logs.tf/1867410
https://demos.tf/120979

Week 3

https://pan.tf/matches/363
https://demos.tf/124937
http://logs.tf/1877758

Week 4
https://pan.tf/matches/418
http://logs.tf/1883017
https://demos.tf/126974

Week 5

https://pan.tf/matches/483
http://logs.tf/1889411
https://demos.tf/129329
B4nny Merc for Top Team Note: Not related to our team but b4nny is a popular player and the top team is playing.

Week 7

https://pan.tf/matches/607
http://logs.tf/1899194
https://demos.tf/133012