Skip to content
Backend EngineerCharleston, South Carolina

JiafanLin

Backend engineer working on distributed data systems — change-data-capture pipelines, streaming ingestion, and the infrastructure that keeps them honest under concurrent load.

CurrentlyBackend Engineer at Gnosis Freight, Charleston
Selected Work
01

Selected Work

01

Distributed AI Inference Platform

A GPU inference platform where interactive users and an autonomous bot share one backend behind a single auth path, with jobs, blob storage and rate limits partitioned by request source.

Internal traffic rides the same credit ledger under a non-billable flag, so bot usage stays observable without distorting revenue — and concurrent dispatch across RunPod clusters removed the serialization bottleneck that had forced generation requests to queue behind one another.

PythonPulumiAzureRunPodGitHub Actions
02

DiaryPure

An iOS app pairing a private journal with consent agreements that both parties sign by finger on the device itself.

Built local-first on SwiftData — entries and captured signatures persist on-device rather than through a backend, leaving Sign in with Apple as the only external identity dependency.

SwiftSwiftUISwiftData
03

Bubble Bobble

A physics-driven arcade platformer running on a hand-rolled game loop — no engine, no physics library. Playable here on the site.

JavaScriptCanvasNext.js
02

About

Profile

I build the parts of a system that have to stay correct when everything is happening at once — change-data-capture pipelines, streaming ingestion, rate limiters, ETL jobs that reconcile the same records over and over without drifting. Most of my work lately has been in Go and Python at Gnosis Freight, on container logistics data that is late, duplicated, and occasionally wrong.

I came to software through games — taking apart the systems behind them long before I could build one — and that is still how I approach a problem: figure out the rules, then find where they leak. There is a platformer on this site I wrote without an engine for exactly that reason.

Outside work I hike. It remains the only reliable way I have found to think about a problem without a screen in front of me.

Experience

Jan 2026 — Present

Charleston, SC

Gnosis Freight

Backend Engineer

  • Cut watchlist load time from two minutes to 500ms by replacing live joins with materialized views, driven by a compiler that turns each grid view into a CDC handler consuming Debezium events off a PlanetScale VStream on ECS Fargate.
  • Built a RabbitMQ task execution framework inside Gantry, the internal CLI, letting engineers enqueue carrier scrapers and ingestion jobs from an arbitrary JSON payload and route them to dedicated workers.
  • Own the scheduled AWS Lambda ETL pipelines reconciling tens of thousands of container records per run across 12+ live terminals, with idempotent add/delete logic and a database trigger enforcing demurrage-billing exemptions.
  • Architected a distributed token bucket rate limiter for internal FastAPI routers, using Redis counters and Lua scripts for atomic check-and-decrement under concurrent load.
  • Built an email-attachment indexer exposed as an LLM agent tool, cutting two to three hours of manual lookup out of repetitive support tickets each week.
GoPythonFastAPIDebeziumRedisAWS

Jul 2024 — Dec 2025

Omaha, NE

Gallup

Software Developer

  • Built a Spark Structured Streaming pipeline consuming Kafka events, parsing nested JSON payloads and aggregating metrics over a ten-minute sliding window.
  • Ingested user interaction events from web and mobile frontends through Kafka, partitioned by event type and user region, sustaining more than 1,000 events per minute.
  • Synced 400k records from Stripe into MySQL and cut nightly ETL runtime by half using composite indexes and parallel retrieval by unique identifier.
  • Developed a DynamoDB transactional service using optimistic locking with version-number attributes and condition checks to hold consistency under concurrent writes.
KafkaSparkMySQLDynamoDB
Education

2022 — 2024

Seattle, WA

Northeastern University

M.S. Information Systems

2018 — 2022

Terre Haute, IN

Rose-Hulman Institute of Technology

B.S. Computer Science & Data Science

Cum Laude · Dean's List, four terms

Toolkit

Languages

  • Go
  • Python
  • Java
  • C#

Backend

  • FastAPI
  • .NET / ASP.NET
  • Node.js

Data & Streaming

  • PostgreSQL
  • MySQL
  • Redis
  • Kafka
  • Debezium

Cloud & Infra

  • AWS
  • Azure
  • Pulumi
  • GitHub Actions

Also worked with

RustC++ReactNext.jsBlazorDjangoFlaskSparkRabbitMQDynamoDBMongoDBNeo4jOracleDatadogSplunk