Jason Magee

Automating WinSCP to Deploy a Site Part 2

Sep 10 2022

In the last post I’d made WinSCP script to copy a local folder to a remote folder to deploy my website. I’ve made some improvements since. Instead of using put to copy the files across I’ve used synchronize to do so.

synchronize remote -criteria=size C:\Path\To\New\Site\

synchronize basically makes one folder the same as the other. You could switch out remote and use local to download changes from the remote. The critera part tells how we compare files between local and remote. As I’m using Hugo, the timestamps get binned when the site is compiled so I only have the sizes to go by.

Another issue I’ve had is with Hugo itself not copying assets that aren’t directly used (e.g. images). So I’ve bunged an xcopy together with the above to make this handy batch file.

Wheel in jasemagee.bat

xcopy /E /i /y C:\Path\To\Site\assets\img C:\Path\To\Site\public\img

"C:\Program Files (x86)\WinSCP\WinSCP.com" /log="C:\Path\To\WinSCP.log" /ini=nul /script="C:\Path\To\Sync website.txt"

Automating WinSCP to Deploy a Site

Sep 9 2022

This is just a snippet of code to automate the deployment of a website (Hugo, Jeykll, etc.) by using WinSCP. I’m using WinSCP because I need WebDAV. I want the script to connect to remote, change to the websites directory and upload the new website version from a folder (overwriting everything in the process) before disconnecting. Here is the script.

open davs://user:pass@webdav.site.com/

cd /path/to/website/directory

put C:\Path\To\New\Site\*

close

exit

This was the first attempt at running the script. It worked but the terminal did not wait for the command to complete, which wasn’t ideal. The only way to confirm it had finished was by reviewing the log file.

"C:\Program Files (x86)\WinSCP\WinSCP.exe" /log="C:\Path\To\Script\WinSCP.log" /ini=nul /script="C:\Path\To\Script\Deploy website.txt"

WinSCP also provides a .com file for scripting, which will cause the terminal to wait.

"C:\Program Files (x86)\WinSCP\WinSCP.com" /log="C:\Path\To\Script\WinSCP.log" /ini=nul /script="C:\Path\To\Script\Deploy website.txt"

Git Dubious Ownership

Aug 31 2022

I’ve just setup a new computer and have been running into this dubious message when using git.

fatal: detected dubious ownership in repository at 'E:/seeker'

 To add an exception for this directory, call:
        git config --global --add safe.directory E:/seeker
 
 Set the environment variable GIT_TEST_DEBUG_UNSAFE_DIRECTORIES=true 
 and run again for more information.
 

Note that tools using git (VS Code) may not work or give any feedback about this fatal error. You have to use the terminal to get at it.

If you DuckDuckGo the message sites will tell you to add the directory as a safe place or turn off this security feature completely in git. This seems a bit silly. The error is telling you that the directory is not owned by the current logged-in user of the machine. I keep my development code on its own SSD which I’ve migrated to the new computer and so Windows had no idea who the owner actually is.

To fix this the best thing you can do is simply change the owner of the directory.

  1. Right click the folder
  2. Click Properties
  3. Go to the Security tab
  4. Click the Advanced Button
  5. At the top on the ‘Owner:’ item, press the Change text at the end
  6. Type part of your username into the name field, Click ‘Check Name’
  7. Ok, ok, ok, ok, etc.
  8. No security issue for git anymore
  9. Make cuppa

Trenchbroom Half-Life Setup

Jun 20 2022

I’ve had an itch lately to make a map for Quake or Half-Life so after a bit of research discovered Trenchbroom which does both. It seems to be highly regarded amongst Quake mappers and has ’experimental’ support for Half-Life. After an hour of research I couldn’t find an easy article to set it up for Half-Life so went down the rabbit hole enough to figure it out.

After more research people seemed convinced Trenchbroom was fine for Half-Life mapping but Trenchbroom does say Half-Life support is experimental so I was a bit suspicious.

The first port of call was checking out how you’d setup Hammer (the original Half-Life map editor) so this excellent site came up quickly for general GoldSrc (name of the Half-Life engine) modding and mapping goodness. All beginners guides say you should use Vluzacn’s ZHLT to build your maps. Looking up ZHLT started to ring some bells from the last time I tried to do Half-Life mapping (when I was a young person). You pass your designed map to a series of tools to “compile” it into an actual game map. These are…

  • HLCSG - makes solid geometry by taking your map and making polygons from the brushes
  • HLBSP - makes the BSP (binary space partition) node tree
  • HLVIS - Generates visibility flags for the polygons (what the player can see)
  • HLRAD - Generates lighting

Looking into Trenchbroom you’ll find similarish tools pre-filled in for Quake, so a bit of fiddling around and I made it work using this.

Image

Here we export the map (in case we haven’t saved it), run the tools above with some parameters that point to the exported map and copy the files into Half-Life’s valve/maps folder (would do different folders for mods).

Next, we setup the Engine simply telling it how to find hl.exe

Image

The game also needs to know what map we want to run (although you can run it via the console) so we use +map XXX and -dev (just enables dev mode).

Image

The texture WADS can be added at the bottom right in the ‘Texture Collections’ area. You’ll find the WAD files in the Half-Life directory (you’ll want at least halflife.wad). A prompt will ask how you want to link the WAD, I went with Absolute but do whatever works for your setup.

Image

Couple of issues

Models do not appear in the editor (e.g. you’d expect a scientist model instead of a green box ideally). Not a big deal.

Image

The properties for entities are not correct and/or auto-populated. I had to manually had Brightness, Appearance and Quadratic. The pre-populated _light value was wrong, so I changed it to 255. You’ll need to dig out the properties you need and add them manually when dealing with entities.

Image

Ultimately, I really like Trenchbroom and will spend some time learning it.

Garmin FIT Files

Jun 8 2022

As I little bit of a side project I recently touched my toes into the world of sports activity data by reading some data from my Garmin watch.

Garmin uses a binary file format called FIT which is a type of GIS file containing geographical data alongside sport data points for whatever activity you're doing.

The FIT file itself wouldn't be hard to parse by hand but there is no point doing so as Garmin provide an SDK to read the data into memory. Once the file is parsed, you can easily export to CSV or do some data analysis with it.

I wrote a small C# app to read any single FIT file and output a bunch of diagrams. Below I've fed the GPS data into OpenStreetMap using the awesome Leaflet library to plot my recent walk. I've also done a couple of charts using Highcharts. Not bad for 2 hours work!

My main takeaway from this small project is suprise at the lack of quality alternatives to Strava. Obviously there is a lot more to Strava (comparisons, analysis, etc.) but with very little effort I've made a dumb activity page.

Six Months Being a Vegetarian

May 31 2022

There isn’t a great deal to say.

I became vegetarian at the beginning of the year, with a vague plan to only eat meat at special occasions but ultimately didn’t. After a month away from meat, I just didn’t fancy it anymore. The whole thing has been very easy and uneventful.

There are only a few things I can think of really mentioning:

  • Guernsey has some restaurants with absolutely naff selections of meat free food. Most have been fine but I’ve been suprised at some well known places.
  • It is sometimes hard to figure out if something is vegetarian. I imagine this was a much bigger ordeal in the past as most things are labelled now. If it’s not though, it can be tough.
  • Some of the things you wouldn’t think contain meat do and it’s pretty disgusting when you look into it. Cheddars, Lilt and Cadbury’s Chocolate Trifle to name a few odd ones. Like, how do you get meat in these things? It’s a bit mad.
  • Chinese takeaways (other than China Red) seem to largely suck for vegetarians.
  • I find the obsession with making vegetable products look and taste like meat quite strange. Vegetables are pretty nice by default, I’m not sure why we need it to taste like sausage.

Disclaimer: I have not been checking alcohol.

26 Year Old Quake Lighting on Hue

Mar 27 2022

Here’s a short video demonstrating a small project I recently worked on. The objective was to take the 26 year old light flickering code from the original Quake (also used in Half-Life, Half-Life 2, Portal 2 and Half-Life Alyx). Last year it made the news after some folk synced up videos of the lighting in each game and found the flickering matched exactly.

26 Year Old Quake Lighting on Hue

Converting the code started by pulling the original lighting patterns from the Quake Source Code and converting it into C# for my little WinForms app.

private readonly Dictionary<string, string> _lightDict = new Dictionary<string, string>
{
    { "normal", "m" },
    { "FLICKER (first variety)", "mmnmmommommnonmmonqnmmo" },
    { "SLOW STRONG PULSE", "abcdefghijklmnopqrstuvwxyzyxwvutsrqponmlkjihgfedcba" },
    { "CANDLE (first variety)", "mmmmmaaaaammmmmaaaaaabcdefgabcdefg" },
    { "FAST STROBE", "mamamamamama" },
    { "GENTLE PULSE 1", "jklmnopqrstuvwxyzyxwvutsrqponmlkj" },
    { "FLICKER (second variety)", "nmonqnmomnmomomno" },
    { "CANDLE (second variety)", "mmmaaaabcdefgmmmmaaaammmaamm" },
    { "CANDLE (third variety)", "mmmaaammmaaammmabcdefaaaammmmabcdefmmmaaaa" },
    { "SLOW STROBE (fourth variety)", "aaaaaaaazzzzzzzz" },
    { "FLUORESCENT FLICKER", "mmamammmmammamamaaamammma" },
    { "SLOW PULSE NOT FADE TO BLACK", "abcdefghijklmnopqrrqponmlkjihgfedcba" },
    { "testing", "a" }
};

From there I simply added a ComboBox with the options, a Setup button (to make a connection with my Hue Bridge) and a Start/Stop button to control the flickering.

Image

The characters represent the flickering pattern, e.g. ‘mmmaaaabcdefgmmmmaaaammmaamm’ is ‘Candle (second variety)’ which ultimately gets converted into a number between 0 and 255 which is sent to my study Hue light as a brightness value allowing it to work with any colour.

It turns out that it’s hard to make a decent quality video when you want to show something on the screen and room lighting!

Playing Duke Nukem 3D in 2022

Mar 15 2022

Duke Nukem 3D was released in 1996 and sits amongst the top first person shooters of its time alongside games like Quake, Doom, etc. It is also known for the game engine under the hood called ‘Build’ which has been used as recently as 2019 for Ion Fury.

Playing it in 2022 turns out to be straight-forward. All you need is a…

  • Legal copy of the game (GOG, Steam, Disks, etc.) so you have access to the files DUKE3D.GRP and DUKE.RTS files
  • Download EDuke32 , a source port of the original and extract it
  • Optionally, the Duke Nukem 3D: High Resolution Pack also extracted (but bin their shareware versions of DUKE3D.GRP and DUKE.RTS)

And Voilà, that’s it!

Duke32 away.

Image
Image
Image
Image

Aged pretty well gameplay wise.

Microsoft SQL Loading Large Dataset

Mar 13 2022

Lately I’ve been loading large datasets into Microsoft SQL (MSSQL) and found the quickest way to do so is using BULK INSERT , even from .NET (instead of SqlBulkCopy for example). Chatting to people, it seems not many knew you could load data straight into MSSQL from a file using this, so I thought I’d write it up.

In my example I’ve downloaded a 5-million-row CSV from this neat source and used the following command to load it into my local MSSQL server.

Using this table…

CREATE TABLE [dbo].[load_5mSalesRecords](
	[Region] [varchar](64) NOT NULL,
	[Country] [varchar](64) NOT NULL,
	[ItemType] [varchar](32) NOT NULL,
	[SalesChannel] [varchar](16) NOT NULL,
	[OrderPriority] [varchar](1) NOT NULL,
	[OrderDate] [varchar](16) NOT NULL,
	[OrderID] [int] NOT NULL,
	[ShipDate] [varchar](16) NOT NULL,
	[UnitsSold] [varchar](8) NOT NULL,
	[UnitPrice] [numeric](6, 2) NOT NULL,
	[UnitCost] [numeric](6, 2) NOT NULL,
	[TotalRevenue] [numeric](10, 2) NOT NULL,
	[TotalCost] [numeric](10, 2) NOT NULL,
	[TotalProfit] [numeric](10, 2) NOT NULL
) ON [PRIMARY]
GO

followed by the BULK INSERT…

BULK INSERT
dbo.load_5mSalesRecords -- target table
FROM
'C:\Users\Jason\Desktop\5m Sales Records.csv' -- source file
WITH
(
TABLOCK, MAXERRORS = 0
, DATAFILETYPE='char'
, CODEPAGE='65001' -- Unicode (UTF-8)
, FIELDTERMINATOR = ',' -- csv so ,
, FIRSTROW=2 -- ignore header row
)
SELECT @@ROWCOUNT

You’re able to load 5 million records in 20 seconds into the table.

Now, the gotcha (isn’t there always!) is that the file is relative to the SQL server and not the machine you’re running the command on. Which seems obvious when you say it but it does mean that if you’re writing an app that makes the file then both systems need to be able to access a shared folder. This is also true on Azure, but you’re able to use Azure Blob storage to get around it.

Count Castle Tilset

Mar 11 2022

I’m releasing the tiles I created for Count Castle for anyone to use under the CC0 licence. It would be nice if credited but I’m easy.

Count Castle Tileset

Image
Count Castle Tileset