Warning! This blog post is full of spoilers.
I spent my free time in the last three evenings trying to figure out what causes the bug in Emogene Takes a Lover where Emogene Cabot does not travel to the Charles View Amphitheater as expected, breaking the Cabot House quest line. I have come to the conclusion that it won't be possible to fix this until the CK is released, but I did discover a lot which contradicts all of the current popular theories out there. Short version: there is no one single cause to this bug. It appears to be a combination of different factors, but finding it is difficult with the tools we have available at this time. I think I'm on to something, but I don't know how much more time I want to waste on fixing Bethesda's buggy software.
This is going to get a little technical, but may be helpful to other modders who maybe can look at it in a fresh way and see something I missed. There may be several edits to this, so if you are interested, come back later to see what else I have found...
How I Tested
In order to ensure this was a clean test, I started a fresh game and immediately headed to undertake this quest line, right out of Vault 111. Purely vanilla game, with no mods whatsoever. (I also tested on a save game which was almost entirely through the main quest line in case there was some condition which required one of those quests to be completed in order to avoid this bug.) If you want to duplicate my results, you can use "tmm 1" to enable all map markers, which saves a lot of time.
There are three quests involved. Special Delivery (503b8), Emogene Takes a Lover (503b9), and The Secret of Cabot House (22a00). The third is actually an overview quest, which starts as soon as you start the game, but won't create Pip-Boy entries until you have completed the second quest. (From here on out, unless I say otherwise, any references to quest stages or "the quest" actually refer to The Secret of Cabot House, since most of the work is actually done there.) So as the quest stages in each of the first two quests get set, a corresponding stage is being silently set in the third. When Emogene doesn't appear as a prisoner at the amphitheater, you are stuck at quest stage 700 and stage 65 for Emogene Takes a Lover. That is just the symptom, though. The issue is that a script which was supposed to run earlier is not triggering; this script should enable the old Emogene NPC (REF ID 47ec4) and move her to her "prison cell" in the amphitheater. So the question is, what are the conditions which cause this script to run or not run?
First, understand that the base NPC for old Emogene (there is also a young Emogene, but that doesn't factor into this yet) does not provide her with any packages except ones which govern her behavior in Cabot House. In other words, even if you manually enable her, she will just wander around Cabot House, regardless of where you are in the quest line. Instead, the quest provides an alias (#6) to her Base ID (47ebe) which adds a few additional packages which govern her behavior for her imprisonment (19BEF, BA5AB, C9455, 19DEF3, 52500). The last two packages are important since one causes her to travel to the amphitheater at a certain point in the quest and the other is a ForceGreet, which in theory should start the dialog regarding her imprisonment and the serum. The bug occurs because this alias is never enabled.
A Partial Mod Fix
I was able to create a mod which enabled Emogene and added the missing packages to her and it worked to a point. The only issue was that in the first quest, she could be seen wandering the house, but with a slight tweak, she would immediately head to the amphitheater once the Sole Survivor headed towards the asylum. This worked 100% of the time. But although she would always show up at the amphitheater, the missing dialog would still not trigger, because its' conditions point to the alias rather than her base ID (and FO4Edit doesn't currently allow creating overrides for dialog records). This would have been a dirty fix had it worked, but I was so close.
Testing Quest Stages
So why is the quest not enabling this alias all/most of the time? The most popular theory out there is that keeping the serum (stage 555 not set) in the first quest causes this bug. I can absolutely guarantee this is not the cause (at least not on its own, as the bug seems to be the result of several conditions). You can test this by giving the serum to Edward Deegan immediately when he asks for it and verify stage 555 is set in the console with "sqs 22a00" (you could also check the appropriate stage in Special Delivery, but those quest stages have no bearing on Emogene Takes a Lover; the overview quest is all that matters here). Whether you give the serum or not, you will likely encounter this bug. Stage 557 is only set if you try to keep the serum but get caught lying (I tried this both ways as well).
If you follow the most logical progression of this quest line and use sqs, you will see that most of the quest stages are getting set normally. So if it was as simple as a quest stage being set/unset (as most players are theorizing), we need to see which stages are optional depending on your choices. Here is a list of all quest stages (through the Emogene rescue anyway), with descriptions and which you may be able to skip (technically you can skip some that I don't list as optional, like you could skip Parsons and go straight to the raiders, but that has no bearing on the Emogene issue). Note that it is possible to go straight to the amphitheater and rescue Emogene once you receive that quest (skip stages 605-700), but from my testing, if the bug is going to happen she won't be there no matter what you do at this point:
|20||Player entered The Third Rail||Yes|
|30||Player knows Lorenzo is a prisoner||Yes|
|32||Player knows about Cabot immortality||Yes|
|50||Start conversation at Cabot House||No|
|100||Edward invited player to Cabot House||Yes|
|150||Edward told player to talk to Jack||Yes|
|250||Player let inside Cabot House||No|
|300||Intro scene in house 1st time||No|
|310||Trigger Jack's experiment explosion||Yes|
|320||Edward starts serving drinks||No|
|325||Edward finishes serving drinks||No|
|340||Update objective marker to point to Jack (only if Edward invited you)||Yes|
|350||Start Jack's intro scene||No|
|360||Jack's intro finished||No|
|400||Objective to Parsons asylum||No|
|405||Player arrives at Parsons||No|
|410||Player learns what the package is||No|
|420||Told captain at Parsons who you are||No|
|450||Objective to raider camp||No|
|500||Objective to return to Cabot House||No|
|510||Player enters Cabot House again||No|
|520||Start argument between Jack and his mother||No|
|550||Package delivered (seems to get set whether you give serum or not)||No|
|555||Give serum to Edward||Yes|
|557||Player caught lying about serum||Yes|
|560||Player gets paid for job||No|
|580||Jack explains about Emogene||No|
|600||Objective to The Third Rail||No|
|605||Whitechapel Charlie points to Magnolia||Yes|
|610||Player talks to Magnolia||Yes|
|650||Learn Emogene is at amphitheater||Yes|
|700||Learn Emogene is locked up||Yes|
|705||Emogene force greets||No|
|710||Ask Emogene about her age||Yes|
|720||Player gives Emogene serum||Yes|
|800||Return to Cabot House||No|
So I made a spreadsheet of every single combination of options for both quests and proceeded to test them one by one. 100% of the time, the bug occurred on a new clean game! Without fail! And I do mean I tested every combination of options. Hundreds of them. For example, visiting The Third Rail first, obtaining the quest from Edward or via speech check at the house, returning the serum, etc. The only tests where the bug did not occur was on certain saves at various points in the game, proving there is some external factor involved.
The only anomaly I could find is that quest stages 30 and 32 are difficult to set before going after Emogene. In theory, these should be set by exhausting all dialog with Jack, questioning him about the asylum and his dad's age, passing speech checks, etc. But whether I did this before going to the asylum the first time or after returning and before going after Emogene, these stages never were set without console commands. Others have said that exhausting the dialog with Jack may prevent this bug, so they may be on to something. Only on one test was I able to get these stages set without the console and it was on a save that was well into the game. This tells me that more than just Jack's dialog is required to set those stages. But I can't say for sure that not setting them is what causes the bug. I can say that if I set them manually with the console (regardless of where I was in the quest), the bug still occurred.
I am convinced now that there is a combination of at least two conditions which are needed to trigger the quest to enable the aliased Emogene. One of them may be exhausting the dialog with Jack, but there is more to it. Debugging quests, dialog, and scripts is a nightmare with our current tools, so hopefully I can take another look at this once the CK is released. But don't waste your time thinking it is something as simple as just one action (such as giving serum or talking to Jack). As with similar bugs we saw in Fallout 3, there is probably some strange condition which relies on an unrelated quest which causes this bug to occur.