The crash spawner and how it worked

[videogamesm12]

Alternative title: "Why Video doesn't have a girlfriend"​

What was the exploit?​

The crash spawner was an exploit which took advantage of a bug in how mob spawners rendered entities to crash anyone who went within its render distance, effectively creating a literal "crash chunk". When mob spawners are rendered in the game, the client creates what is effectively a "mini" version of the entity to show it rotating inside the block. Mojang made two mistakes with Paintings and Item Frames in particular where they forgot to specify defaults in the code the spawner uses to spawn the "mini" entity, which causes the game to crash upon attempting to render said "mini" entity with a NullPointerException. The bug had been present in the game since mob spawners were introduced all the way back in June 2010 with the release of infdev version 20100625-1917, but wasn't known about for several years. It was patched by Mojang with the release of 1.9.

It first surfaced on the server as early as July 2015 after being used to create crash chunks in the adminworld, but was later used heavily by various exploiters starting in July and ending when the server updated to Minecraft 1.9 in early 2016. While the exploit itself has been known since then, it was never properly documented until now because 1) exploit research was a lot more primitive and less developed back then and 2) it's a lot harder to debug crashes in older versions of the game due to the lack of properly supported modern modding tools that support these ancient versions. As an exercise, I decided to spend an afternoon debugging this exploit and figuring out how it works after not properly understanding it for so long.

This technical explanation was written with the codebase of version 1.8 in mind, and should not be applied to any versions newer than it as the game is likely structured very differently internally now.

How exactly did it work?​

Before we begin, I need to explain something technical. In Java, almost everything has something called a Constructor, which serves effectively as a form with fields that anyone wishing to create said an instance of said object needs to fill out properly. Once filled out, your computer generates the requested object using the information provided to it and spits it out. You can have multiple constructors for a given object type and if you are extending off another object type's code will inherit that object type's constructors. Got it? Good.

Everything you see in the game has a dedicated renderer for it, with the mob spawner being no exception. Every frame, the game goes through and draws things like the world, the HUD, and any open GUI in a series of steps. When it's a mob spawner's turn to be rendered, it gets the entity it's supposed to be for and checks to see if its "mini" entity exists and if it doesn't, it calls a piece of code that looks for a specific constructor that every entity type is expected to have which only asks for the world it's supposed to be in and creates an instance of that entity with that constructor. This is normally not an issue as most values should just have defaults set with that constructor, but with Paintings and Item Frames it's an entirely different story, which I'll explain in just a moment. Once the entity instance has been created, the game then delegates the task of rendering the entity to the renderer for that specific type of entity.

If the entity is a Painting​

Internally, the client has 3 different constructors for Paintings as shown by this excerpt of code (remapped for readability):

public class PaintingEntity extends AbstractDecorationEntity {
    public PaintingEntity.PaintingMotive type;

    public PaintingEntity(World world) {
        super(world);
    }

    public PaintingEntity(World world, BlockPos blockPos, Direction direction) {
        super(world, blockPos);
        List<PaintingEntity.PaintingMotive> list = Lists.newArrayList();

        for (PaintingEntity.PaintingMotive paintingMotive : PaintingEntity.PaintingMotive.values()) {
            this.type = paintingMotive;
            this.setDirection(direction);
            if (this.isPosValid()) {
                list.add(paintingMotive);
            }
        }

        if (!list.isEmpty()) {
            this.type = (PaintingEntity.PaintingMotive)list.get(this.random.nextInt(list.size()));
        }

        this.setDirection(direction);
    }

    @Environment(EnvType.CLIENT)
    public PaintingEntity(World world, BlockPos blockPos, Direction direction, String string) {
        this(world, blockPos, direction);

        for (PaintingEntity.PaintingMotive paintingMotive : PaintingEntity.PaintingMotive.values()) {
            if (paintingMotive.name.equals(string)) {
                this.type = paintingMotive;
                break;
            }
        }

        this.setDirection(direction);
    }
}

This may seem overwhelming at first, but I'll break it down for you:

1. A constructor asking for just the world and nothing more. This is only called indirectly by the mob spawner code mentioned above. It simply calls the constructor it inherited from AbstractDecorationEntity which just sets the world it's for and sets the bounding box.
2. A constructor asking for the world, the block position it's supposed to be located at, and the direction it's facing. This gets used when you place a Painting down. It loops through every type, filters out anything that wouldn't fit in the area it's in, and then adds it to a list. It then picks a random type from that list and sets that as the type to use. Finally, it sets the direction the Painting is supposed to be facing.
3. A constructor asking for the world, the block position it's supposed to be located at, the direction it's facing, and the exact picture the Painting is of. This gets used when the client receives a message from the server telling it that a specific type of Painting spawned at a series of coordinates facing a certain way. It first calls code for the constructor above and then after that's done, it then checks to see if the type the server told it about exists and if it does, it sets that to be the type of Painting to use. Afterwards, it sets the direction again.

So, what happens when you try to fill out the first constructor? Well, a Painting is still generated, but its type and directions aren't set. In Java, when a non-final field isn't set, it will fallback to setting it to null, meaning nothing. What we have here is a trap, as while we do have a technically valid Painting entity, anything that attempts to access properties of the direction or type will cause Java to throw an error because you can't access information about something that doesn't exist.

Moving on, we now get to the rendering code. I'm not going to explain how the game actually renders it aside from giving a brief overview because it's both complicated and irrelevant to the nature of the exploit.

public void render(PaintingEntity paintingEntity, double d, double e, double f, float g, float h) {
    GlStateManager.pushMatrix();
    GlStateManager.translate(d, e, f);
    GlStateManager.rotate(180.0F - g, 0.0F, 1.0F, 0.0F);
    GlStateManager.enableRescaleNormal();
    this.bindTexture(paintingEntity);
    PaintingEntity.PaintingMotive paintingMotive = paintingEntity.type;
    float i = 0.0625F;
    GlStateManager.scale(i, i, i);
    this.renderPainting(paintingEntity, paintingMotive.width, paintingMotive.height, paintingMotive.textureX, paintingMotive.textureY);
    GlStateManager.disableRescaleNormal();
    GlStateManager.popMatrix();
    super.render(paintingEntity, d, e, f, g, h);
}

The game first performs some prelimitary work with things like calculating offsets and applying some basic graphical transformations. Then, it sets a variable (sort of like a "shortcut" in a sense) to the value of the Painting entity's type. In our case, this just sets the variable to null because that's what the type was set to earlier. After performing some quick scaling, the game then attempts to call a method related to actually rendering the Painting with it, the type's wid-

View attachment Windows XP Critical Stop.ogg​

Oh, shit. Before it can even call the method, Java intervenes and throws up an error because the game just tried to access a property of a nonexistant type, resulting in a game crash.

If the entity is an Item Frame​

Item Frames have two constructors as shown by this excerpt of code (remapped for readability):

public class ItemFrameEntity extends AbstractDecorationEntity {
    private float setDropChance = 1.0F;

    public ItemFrameEntity(World world) {
        super(world);
    }

    public ItemFrameEntity(World world, BlockPos blockPos, Direction direction) {
        super(world, blockPos);
        this.setDirection(direction);
    }
}

Things are a lot simpler this time around as the game loads data like the item separately and only when it needs to do so. Here's a quick breakdown nonetheless:

1. A constructor asking for just the world and nothing more. Like Paintings, this is only called indirectly by the mob spawner code. It calls the constructor it inherited from AbstractDecorationEntity which sets the world it's for and the bounding box.
2. A constructor asking for the world, the block position it's supposed to be located at, and the direction it's facing. This is called when you either place down the Item Frame or the server sends a message to the client telling it that an entity of that specific type spawned in.

The same mistake happens with the first constructor with pretty much the same result: a technically valid Item Frame with no direction. Relatable.

Moving onto the rendering code again, you know the deal already:

public void render(ItemFrameEntity itemFrameEntity, double d, double e, double f, float g, float h) {
    GlStateManager.pushMatrix();
    BlockPos blockPos = itemFrameEntity.getTilePos();
    double i = (double)blockPos.getX() - itemFrameEntity.x + d;
    double j = (double)blockPos.getY() - itemFrameEntity.y + e;
    double k = (double)blockPos.getZ() - itemFrameEntity.z + f;
    GlStateManager.translate(i + 0.5, j + 0.5, k + 0.5);
    GlStateManager.rotate(180.0F - itemFrameEntity.yaw, 0.0F, 1.0F, 0.0F);
    this.dispatcher.textureManager.bindTexture(SpriteAtlasTexture.BLOCK_ATLAS_TEX);
    BlockRenderManager blockRenderManager = this.field_8000.getBlockRenderManager();
    BakedModelManager bakedModelManager = blockRenderManager.getModels().getBakedModelManager();
    BakedModel bakedModel;
    if (itemFrameEntity.getHeldItemStack() != null && itemFrameEntity.getHeldItemStack().getItem() == Items.FILLED_MAP) {
        bakedModel = bakedModelManager.getByIdentifier(this.field_11112);
    } else {
        bakedModel = bakedModelManager.getByIdentifier(this.field_11111);
    }

    GlStateManager.pushMatrix();
    GlStateManager.translate(-0.5F, -0.5F, -0.5F);
    blockRenderManager.getModelRenderer().render(bakedModel, 1.0F, 1.0F, 1.0F, 1.0F);
    GlStateManager.popMatrix();
    GlStateManager.translate(0.0F, 0.0F, 0.4375F);
    this.renderItem(itemFrameEntity);
    GlStateManager.popMatrix();
    this.renderFrame(
        itemFrameEntity,
        d + (double)((float)itemFrameEntity.direction.getOffsetX() * 0.3F),
        e - 0.25,
        f + (double)((float)itemFrameEntity.direction.getOffsetZ() * 0.3F)
    );
}

The game performs a lot of rendering here code here related to the items, but the part that's relevant to us is at the very end. After rendering the item and popping a matrix (I don't exactly know what that means but it sounds really cool), the game then attempts to call an internal method for rendering the frame with it, the frame's X coordinates + the product of multiplying the frame's direc-

View attachment Windows XP Critical Stop.ogg​

Yeah, same thing. Java intervenes and throws up an error because the game just tried to get a nonexistant direction to do something, resulting in a game crash.

How do I know if my game crashed because of this exploit?​

Crash reports are typically intended for the game's developers to troubleshoot the issue, but you do not need to be a 300 IQ genius to read it. Crash reports typically contain a bunch of useful information that is crucial for identifying the cause of the crash such as the stacktrace (which is basically a paper trail of steps the game was taking before it encountered a problem) and sometimes whatever caused the game to trip over itself. Here is a crash report that generated as a result of encountering this exploit in the wild, which I will show how I know it was caused by this exploit:

Spoiler: Full crash report, fetched from an old thread on the ProBoards forum

---- Minecraft Crash Report ----
// But it works on my machine.

Time: 7/26/15 4:14 PM
Description: Rendering entity in world

java.lang.NullPointerException: Rendering entity in world
    at bjr.a(SourceFile:35)
    at bjr.a(SourceFile:16)
    at biu.a(SourceFile:295)
    at biu.a(SourceFile:282)
    at bhh.a(SourceFile:28)
    at bhh.a(SourceFile:14)
    at bhh.a(SourceFile:9)
    at bhc.a(SourceFile:114)
    at bhc.a(SourceFile:102)
    at bfr.a(SourceFile:615)
    at bfk.a(SourceFile:1244)
    at bfk.b(SourceFile:1149)
    at bfk.a(SourceFile:1002)
    at ave.av(SourceFile:914)
    at ave.a(SourceFile:325)
    at net.minecraft.client.main.Main.main(SourceFile:124)


A detailed walkthrough of the error, its code path and all known details is as follows:
---------------------------------------------------------------------------------------

-- Head --
Stacktrace:
    at bjr.a(SourceFile:35)
    at bjr.a(SourceFile:16)

-- Entity being rendered --
Details:
    Entity Type: Painting (uq)
    Entity ID: 11
    Entity Name: Painting
    Entity's Exact location: 10.00, 0.00, -8.00
    Entity's Block location: 10.00,0.00,-8.00 - World: (10,0,-8), Chunk: (at 10,0,8 in 0,-1; contains blocks 0,0,-16 to 15,255,-1), Region: (0,-1; contains chunks 0,-32 to 31,-1, blocks 0,0,-512 to 511,255,-1)
    Entity's Momentum: 0.00, 0.00, 0.00
    Entity's Rider: ~~ERROR~~ NullPointerException: null
    Entity's Vehicle: ~~ERROR~~ NullPointerException: null

-- Renderer details --
Details:
    Assigned renderer: bjr@a20898c
    Location: 0.00,0.00,0.00 - World: (0,0,0), Chunk: (at 0,0,0 in 0,0; contains blocks 0,0,0 to 15,255,15), Region: (0,0; contains chunks 0,0 to 31,31, blocks 0,0,0 to 511,255,511)
    Rotation: 0.0
    Delta: 0.7207227
Stacktrace:
    at biu.a(SourceFile:295)
    at biu.a(SourceFile:282)
    at bhh.a(SourceFile:28)
    at bhh.a(SourceFile:14)
    at bhh.a(SourceFile:9)

-- Block Entity Details --
Details:
    Name: MobSpawner // all
    Block type: ID #52 (tile.mobSpawner // ahy)
    Block data value: 0 / 0x0 / 0b0000
    Block location: World: (45911,50,14605), Chunk: (at 7,3,13 in 2869,912; contains blocks 45904,0,14592 to 45919,255,14607), Region: (89,28; contains chunks 2848,896 to 2879,927, blocks 45568,0,14336 to 46079,255,14847)
    Actual block type: ID #52 (tile.mobSpawner // ahy)
    Actual block data value: 0 / 0x0 / 0b0000
Stacktrace:
    at bhc.a(SourceFile:114)
    at bhc.a(SourceFile:102)
    at bfr.a(SourceFile:615)
    at bfk.a(SourceFile:1244)
    at bfk.b(SourceFile:1149)

-- Affected level --
Details:
    Level name: MpServer
    All players: 1 total; [bew['IDoNotCare21'/219, l='MpServer', x=45900.50, y=50.00, z=14612.50]]
    Chunk stats: MultiplayerChunkCache: 145, 145
    Level seed: 0
    Level generator: ID 00 - default, ver 1. Features enabled: false
    Level generator options:
    Level spawn location: 0.00,50.00,0.00 - World: (0,50,0), Chunk: (at 0,3,0 in 0,0; contains blocks 0,0,0 to 15,255,15), Region: (0,0; contains chunks 0,0 to 31,31, blocks 0,0,0 to 511,255,511)
    Level time: 1564584 game time, 30000 day time
    Level dimension: 0
    Level storage version: 0x00000 - Unknown?
    Level weather: Rain time: 0 (now: false), thunder time: 0 (now: false)
    Level game mode: Game mode: creative (ID 1). Hardcore: false. Cheats: false
    Forced entities: 1 total; [bew['IDoNotCare21'/219, l='MpServer', x=45900.50, y=50.00, z=14612.50]]
    Retry entities: 0 total; []
    Server brand: Spigot
    Server type: Non-integrated multiplayer server
Stacktrace:
    at bdb.a(SourceFile:309)
    at ave.b(SourceFile:2293)
    at ave.a(SourceFile:334)
    at net.minecraft.client.main.Main.main(SourceFile:124)

-- System Details --
Details:
    Minecraft Version: 1.8.7
    Operating System: Windows 8.1 (amd64) version 6.3
    CPU: 4x Intel(R) Core(TM) i3-3120M CPU @ 2.50GHz
    Java Version: 1.8.0_25, Oracle Corporation
    Java VM Version: Java HotSpot(TM) 64-Bit Server VM (mixed mode), Oracle Corporation
    Memory: 84042696 bytes (80 MB) / 242696192 bytes (231 MB) up to 1060372480 bytes (1011 MB)
    JVM Flags: 6 total; -XX:HeapDumpPath=MojangTricksIntelDriversForPerformance_javaw.exe_minecraft.exe.heapdump -Xmx1G -XX:+UseConcMarkSweepGC -XX:+CMSIncrementalMode -XX:-UseAdaptiveSizePolicy -Xmn128M
    IntCache: cache: 0, tcache: 0, allocated: 13, tallocated: 95
    Launched Version: 1.8.7
    LWJGL: 2.9.4
    OpenGL: Intel(R) HD Graphics 4000 GL version 4.0.0 - Build 10.18.10.3308, Intel
    GL Caps: Using GL 1.3 multitexturing.
Using GL 1.3 texture combiners.
Using framebuffer objects because OpenGL 3.0 is supported and separate blending is supported.
Shaders are available because OpenGL 2.1 is supported.
VBOs are available because OpenGL 1.5 is supported.

    Using VBOs: No
    Is Modded: Probably not. Jar signature remains and client brand is untouched.
    Type: Client (map_client.txt)
    Resource Packs: []
    Current Language: English (US)
    Profiler Position: N/A (disabled)

From here, we have a few clues as for the cause of the crash:

1. The stacktrace (basically a series of steps the game was taking right before the game melted) at the very top says java.lang.NullPointerException: Rendering entity in world.
   This means that while the game was working on rendering an entity, it tried doing something with something that didn't exist. This issue is not unique to this exploit and can happen in a lot different ways, so it is not exactly a damning clue.
   
   
2. The entity type listed in the "Entity being rendered" section is a Painting or Item Frame.
   Now we're getting somewhere. This means that the entity was one of those two which caused the crash. However, crashes might still be caused the game attempting to render the item within an Item Frame, so it's still not decisive. However, we are getting warmer.
   
   
3. A section titled Block Entity Details is present and mentions a Mob Spawner.
   We're getting pretty hot now. This means that not only was there a block entity involved, but specifically a Mob Spawner. It's becoming less and less likely to be caused by something else at this point.

If all of the clues mentioned before match up with your crash report, then it's safe to say this exploit (or something similar) is causing your game to crash. Now that you know why your game crashed, it's now time for the next steps: resolving the situation. For this, I'm going to explain how you can use more modern solutions to fix the problem and then give a bit of a history lesson by explaining how staff on TF used to handle exploits back in the day when the tools we were given were so primitive.

How should I resolve this?​

We live in an era where the toolings available to the average server owner, server admin, and modders have advanced greatly, and I'd hope that nowadays people are taking a much more surgical approach to resolving exploits. The first thing we need to do is figure out where the problematic spawner is located, and that's where the crash report will come in clutch again, as the "Block Entity Details" section includes an exact set of coordinates as for where the spawner is. In the case of the one listed above, the coordinates of the spawner was located at XYZ 45911, 50, 14605.

Removing the problematic spawner​

If you have WorldEdit installed, you can use the //pos1 and //pos2 command to select that exact block and then replace it with something harmless with the //set command. If it's not installed, you will need to log into the server with a client that isn't affected by the exploit (such as MinecraftConsoleClient or even a newer version of the game) to load the chunk with the spawner and then remove the block manually with the /setblock command or by hand if you're running a newer version of the game. If it worked, you should be able to log into the server again with your regular client without issue.

Figuring out who placed it​

If you had CoreProtect installed at the time of the incident, you should be able to use the /co lookup command to figure out who placed any mob spawners and work backwards from there with this command: /co lookup t:365d r:#global block:spawner. If that doesn't work, you can use the plugin's inspector to manually lookup the history of everything that happened at that exact block. It will list the name of whoever placed the block along with where it happened (if you didn't already know). If you want you can then perform a rollback, but if you were able to remove it before with the mentioned above then this isn't necessary.

How did we resolve this back in the day?​

It's hard to believe now, but back in the day we didn't have administrative tools like CoreProtect. In fact, there was considerable resistance at the time to even add the plugin due to concerns about the potential overhead it may introduce and how large the database could be, and ultimately it wasn't until April 2017 (a whole year after the exploit was patched by Mojang) when the server would have it installed. Our alternative was a combination of the Logstick (a feature of the TotalFreedomMod in which manual block changes were recorded to a database) and reading the latest.log file with the Logviewer.

When we were dealing with this exploit, a solution that we were initially taught to do was to first remove all Mob Spawners within a specific radius of all players using the /ro command and if that didn't work despite our best efforts, we would simply wipe the Flatlands clean and start over. This is just how we did things as the Flatlands used to be considered a temporary world that would reset very frequently.

It's a mob spawner that spawns paintings, which is something that doesn't exist, so your client crashes. If it wasn't in the adminworld, a flatlands reset or /ro 52 will fix it.

Later on, more surgical methods involving alternative clients, WorldEdit, or even vanilla commands like /setblock came up and were utilized to mitigate the problem:

I have, through local testing, confirmed that the crashing players have experienced recently is a rendering issue. The Minecraft Console Client is unaffected by this crash bug, and as such can be used to teleport yourself away from the source of the crashing.

Note that due to this client lacking any rendering, it is totally unaffected by most client crash exploits.

To bypass the crash loop from a mob spawner/painting:
- Log in to the server with this client. (you need to add :28965 to the end of the IP, as this client doesn't support SRV records)
- Teleport away from where you crashed (using a command such as tppos)
- Do /quit to exit the console client
- Log back in with a normal Minecraft client

To figure out who actually did it, we usually performed the aforementioned surgical method of replacing the block with something else and then checking the history of who placed the Mob Spawner happened at that location with either the Logstick or the Logviewer, as suggested by indefinite ban requests of the time. They typically looked like this, both of which were considered acceptable evidence of wrongdoing:

[14:06:08] [Client thread/INFO]: [CHAT] Block edits at (x538, y50, z489):
[14:06:08] [Client thread/INFO]: [CHAT] - Assasain_dude2 placed Mob_spawner

:seen Assasain_dude2
:Player Assasain_dude2 has been offline since 10 minutes.
: - IP Address: [REDACTED]
: - Location: (flatlands, 538, 50, 492)

Client crash log showing crash spawner block location:

-- Block Entity Details --
Details:
Name: MobSpawner // net.minecraft.tileentity.TileEntityMobSpawner
Block type: ID #52 (tile.mobSpawner // net.minecraft.block.BlockMobSpawner)
Block data value: 0 / 0x0 / 0b0000
Block location: World: (538,50,489), Chunk: (at 10,3,9 in 33,30; contains blocks 528,0,480 to 543,255,495), Region: (1,0; contains chunks 32,0 to 63,31, blocks 512,0,0 to 1023,255,511)

[21:49:58] [Server thread/INFO]: Player issued server command: /summon FallingSand 4865 58 242 {Block:mob_spawner,Data:0,TileEntityData:{EntityId:Painting},Time:1}

Conclusion​

Thank you for reading this. I wrote this thread because I've been working on documenting exploits that were historically used against the server during its lifetime for the TF wiki, and the crash spawner seemed like a great historical exploit to research and teach people about, especially because since it was so easy to pull off and yet its root cause wasn't so obvious when it was in its prime. It was also one of the earliest exploits that I still remember and was always mystified by how it actually worked, so it was a fun exercise to read the code and figure out how it ticked. I might do more of these kinds of retrospective threads if the idea is popular enough, so let me know if I should.

 

[zrfv]

moyang still cant code