How I built a 20 step AI workflow for Real Estate Agent lead intake processing

September 13, 2025

Author: Atum

39 min read

AI Workflow for Real Estate Leads
(Click to view full screen)

I created a web app with an AI workflow with code.

The main feature was an AI workflow for real estate agents who have leads emailing them.

Instead of the agent having to manually read and reply to every email, the AI workflow would do that.

The selling point: have dinner with your family or go to sleep with peace of mind knowing that you won’t miss out on any leads because you didn't reply in time.

This article will be about the technical implementation of the AI workflow.

Architectural choices

Quite a standard tech stack. Next.js + Shadcn + Supabase is still undefeated for creating a modern web app with an amazing developer experience.

Nylas is a paid email API service. This allowed me to connect with user's google email accounts easier than if I was doing it myself.

The real issue with email connection was with Microsoft office. I was unable to figure out how to get this set up. Whoever is in charge of Microsoft developer services needs to do a better job.

Google Gemini is the worst AI product I've used. It's the only AI where it actively disagreed with me even when I pointed out how wrong it was, it continued to tell me I was wrong.

Also there were times where the conversation went a bit controversial (talking about contents about a book then transitioning to therapy role play) and it literally deleted the entire conversation history and there was no more context. The safeguards in Google Gemini are ridicuously high. The code produced isn't good either. Anyone shilling for Gemini is a paid actor.

That being said, the API was cheap and there was nothing controversial happening in my AI API calls.

Update about Nylas: I do not recommend them. I tried to cancel my subscription and I wasn't able to do it manually, I had to request to cancel, and after like a month it's still not cancelled. I emailed them several times and they said something about your account will end after 2 more payments. So I had to report them to several government websites and better business bureau and had to start a dispute against them with my credit card company.

Technical implementation

I used AI to generate at least 95% of the code.

My preferred AI tool of choice was and is Cursor agent. This agent has access to your code base so the context is unbeatable in my opinion.

You can copy and paste from Claude or Chatgpt but that's best for high level planning or self contained code. For implementation in your codebase, Cursor is the best I've used.

I work as a vibe coder, feeling my way through the features. I know what is missing, what features are needed, so I can tell Cursor what to do. Implementation isn't my bottleneck, it's knowing what to do and how to fix the bugs. Cursor takes care of the details.

AI workflow

In this section I will be going over the step by step actions the AI workflow takes to do its job.

This does not include general saas coding such as components, layouts, database design, api calls etc...

This is only about the AI workflow.

The code snippets are real code used in the project.

1. Webhook gets triggered

The starting point of the AI flow. The webhook gets triggered when the user gets a new email.

  1. Request gets validated for security.
  2. The email and message is extracted.
const webhookSecret = process.env.NYLAS_WEBHOOK_SECRET;
const signature = request.headers.get("x-nylas-signature");
const body = await request.text();

if (!webhookSecret) {
  console.error("NYLAS_WEBHOOK_SECRET is not set");
  return NextResponse.json(
    { error: "Server misconfiguration" },
    { status: 500 }
  );
}

// Verify signature
const expectedSignature = crypto
  .createHmac("sha256", webhookSecret)
  .update(body)
  .digest("hex");

if (signature !== expectedSignature) {
  console.warn("Invalid Nylas webhook signature");
  return NextResponse.json({ error: "Invalid signature" }, { status: 401 });
}

// Parse and handle the event
let event: NylasWebhookPayload;
try {
  event = JSON.parse(body);
} catch (e) {
  return NextResponse.json({ error: "Invalid JSON" }, { status: 400 });
}

// Handle message.created events
if (
  event.type === "message.created" &&
  event.data?.object?.grant_id &&
  event.data?.object?.id
) {
  const nylasMessage = event.data.object;
  const grantId = nylasMessage.grant_id;
  const messageId = nylasMessage.id;

  try {
    // 1. Extract email content for analysis
    const plainText =
      nylasMessage.body_text ||
      (nylasMessage.body ? stripHtml(nylasMessage.body) : "");

2. shouldProcessEmail is activated

This method calls classifyAndExtract which is an AI api call to check the contents of the email to see if the email should be processed or not.

export async function shouldProcessEmail(emailText: string): Promise<{
  shouldProcess: boolean;
  reason?: string;
  classification?: any;
}> {
  try {
    if (!emailText || emailText.trim().length === 0) {
      return {
        shouldProcess: false,
        reason: "Empty email content",
      };
    }

    // Classify the email content using AI
    const classification = await classifyAndExtract(emailText);

    // Filter out non-real estate emails
    if (classification.intent === "not_interested") {
      return {
        shouldProcess: false,
        reason: "Email classified as 'not_interested'",
        classification,
      };
    }

    if (classification.intent === "other") {
      return {
        shouldProcess: false,
        reason: "Email classified as 'other' (non-real estate)",
        classification,
      };
    }

    // Only process real estate related intents
    const validIntents = [
      "initial_inquiry",
      "price_question",
      "showing_request",
      "follow_up",
    ];
    if (!validIntents.includes(classification.intent)) {
      return {
        shouldProcess: false,
        reason: `Email intent '${classification.intent}' not in valid intents`,
        classification,
      };
    }

    return {
      shouldProcess: true,
      classification,
    };
  } catch (error) {
    console.error("đź”´ Error in initial email filtering:", error);
    // Default to not processing if there's an error
    return {
      shouldProcess: false,
      reason: "Error during email analysis",
    };
  }
}

3. classifyAndExtract is called

  1. This takes the text content of the email.
  2. Creates a function tool for the AI api call.
  3. Calls the Google Gemini api to classify the email, we are only interested in real estate related inqueries.
  4. Returns the intent, entities, and confidence.
  5. If the email is not related to real estate or the intent is not_interested then return false.
  6. If the email should be processed then return the classifcation and true.
export async function classifyAndExtract(
  text: string
): Promise<ClassificationResult> {
  try {
    // Validate input
    if (!text || typeof text !== "string") {
      throw new Error(
        `Invalid input: text must be a non-empty string, got ${typeof text}`
      );
    }

    // Validate API key
    if (!process.env.GOOGLE_AI_API_KEY) {
      throw new Error("GOOGLE_AI_API_KEY environment variable is not set");
    }

    const classificationToolDeclaration: FunctionDeclaration = {
      name: "classify_email_intent_and_entities",
      parameters: {
        type: Type.OBJECT,
        description:
          "Classifies the intent of an email and extracts key entities.",
        properties: {
          intent: {
            type: Type.STRING,
            enum: [...INTENTS],
            description: "The primary intent of the email.",
          },
          entities: {
            type: Type.ARRAY,
            items: {
              type: Type.OBJECT,
              properties: {
                type: {
                  type: Type.STRING,
                  enum: [...ENTITIES],
                  description:
                    "The type of the extracted entity (e.g., 'date', 'location').",
                },
                value: {
                  type: Type.STRING,
                  description: "The extracted value of the entity.",
                },
              },
              required: ["type", "value"],
            },
            description: "A list of all entities found in the email.",
          },
          confidence: {
            type: Type.NUMBER,
            description:
              "A confidence score from 0.0 to 1.0 about the classification.",
          },
        },
        required: ["intent", "entities", "confidence"],
      },
    };

    const response = await ai.models.generateContent({
      model: "gemini-2.5-flash",
      contents: [
        {
          role: "user",
          parts: [
            {
              text: `You are an expert AI assistant for a real estate business. Analyze this email and classify the user's intent and extract key entities.

IMPORTANT: Only classify as real estate related intents if the email is clearly about real estate services (buying, selling, renting, investing in property). 

- "other" = Non-real estate emails (newsletters, spam, personal emails, business unrelated to real estate)
- "not_interested" = People explicitly saying they don't want to be contacted or are unsubscribing
- "initial_inquiry" = First contact about real estate services
- "price_question" = Questions about property prices, values, or costs
- "showing_request" = Requests to view properties or schedule appointments
- "follow_up" = Follow-up questions about previous real estate discussions

Email to analyze:
${text.slice(0, 4000)}`,
            },
          ],
        },
      ],
      config: {
        toolConfig: {
          functionCallingConfig: {
            mode: FunctionCallingConfigMode.ANY,
            allowedFunctionNames: ["classify_email_intent_and_entities"],
          },
        },
        tools: [{ functionDeclarations: [classificationToolDeclaration] }],
      },
    });

    const functionCalls = response.functionCalls;
    if (!functionCalls || !functionCalls[0]?.args) {
      console.error("đź”´ AI analysis failed to return a valid function call.");
      console.error("Response:", JSON.stringify(response, null, 2));
      return { intent: "other", entities: [], confidence: 0.3 };
    }

    const args = functionCalls[0].args as {
      intent: Intent;
      entities: ExtractedEntity[];
      confidence: number;
    };

    // Validate the response
    if (!args.intent || !INTENTS.includes(args.intent)) {
      console.error("đź”´ AI returned invalid intent:", args.intent);
      return { intent: "other", entities: [], confidence: 0.3 };
    }

    return {
      intent: args.intent || "other",
      entities: args.entities || [],
      confidence: args.confidence || 0.5,
    };
  } catch (error) {
    console.error("đź”´ Error in classifyAndExtract:", error);

    // Provide more context about the error
    if (error instanceof Error) {
      if (error.message.includes("Could not load the default credentials")) {
        console.error(
          "đź”´ Google AI authentication error: Please check GOOGLE_AI_API_KEY environment variable"
        );
      } else if (error.message.includes("API key")) {
        console.error(
          "đź”´ Google AI API key error: Please verify your API key is valid"
        );
      } else if (
        error.message.includes("quota") ||
        error.message.includes("rate limit")
      ) {
        console.error(
          "đź”´ Google AI quota/rate limit error: Please check your usage limits"
        );
      }
    }

    // Return a default error state
    return { intent: "other", entities: [], confidence: 0.0 };
  }
}

4. If we should continue processing the email then:

  1. Extract the email.
  2. Find the channel associated with the email (the channel is the connected email).
  3. Get the channel configuration (email templates, lead types enabled, user settings etc...).
  4. Create the contact.
  5. Create or update the email conversation.
  6. Save the message sent.
  7. Begin conversation processing.
const fromEmail = nylasMessage.from?.[0]?.email;
const toEmail = nylasMessage.to?.[0]?.email;

if (!fromEmail || !toEmail) {
  console.error("đź”´ Missing email addresses in message");
  return NextResponse.json(
    { error: "Invalid message format" },
    { status: 400 }
  );
}

const { data: channel, error: channelError } = await supabaseAdmin
  .from("channels")
  .select("*")
  .eq("identifier", toEmail)
  .single();

if (channelError || !channel) {
  console.error(`đź”´ No channel found for Nylas account ${grantId}`);
  return NextResponse.json(
    { error: "Channel not found" },
    { status: 404 }
  );
}

const { data: channelConfig } = await supabaseAdmin
  .from("channel_configurations")
  .select("test_mode_enabled, test_mode_allowed_emails")
  .eq("channel_id", channel.id)
  .single();

// Check test mode configuration
if (channelConfig?.test_mode_enabled) {
  const allowedEmails =
    channelConfig.test_mode_allowed_emails
      ?.split("\n")
      .map((email: string) => email.trim())
      .filter((email: string) => email.length > 0) || [];

  if (!allowedEmails.includes(fromEmail)) {
    console.log(
      `🔵 Skipping email from ${fromEmail} - test mode active, not in allowed list: ${allowedEmails.join(", ")}`
    );
    return NextResponse.json(
      {
        received: true,
        skipped: true,
        reason: `Test mode - only processing emails from: ${allowedEmails.join(", ")}`,
      },
      { status: 200 }
    );
  }
}

let contactId: string;
const { data: existingContact } = await supabaseAdmin
  .from("contacts")
  .select("id")
  .eq("org_id", channel.org_id)
  .eq("email", fromEmail)
  .single();

if (existingContact) {
  contactId = existingContact.id;
} else {
  const { data: newContact, error: contactError } = await supabaseAdmin
    .from("contacts")
    .insert({
      org_id: channel.org_id,
      email: fromEmail,
      name: nylasMessage.from?.[0]?.name || null,
    })
    .select("id")
    .single();

  if (contactError || !newContact) {
    console.error("đź”´ Failed to create contact:", contactError);
    return NextResponse.json(
      { error: "Failed to create contact" },
      { status: 500 }
    );
  }
  contactId = newContact.id;
}

let leadId: string;
const { data: existingLead } = await supabaseAdmin
  .from("leads")
  .select("id")
  .eq("channel_id", channel.id)
  .eq("contact_id", contactId)
  .single();

if (existingLead) {
  leadId = existingLead.id;
} else {
  const { data: newLead, error: leadError } = await supabaseAdmin
    .from("leads")
    .insert({
      org_id: channel.org_id,
      channel_id: channel.id,
      contact_id: contactId,
      source: "email",
      status: "new",
    })
    .select("id")
    .single();

  if (leadError || !newLead) {
    console.error("đź”´ Failed to create lead:", leadError);
    return NextResponse.json(
      { error: "Failed to create lead" },
      { status: 500 }
    );
  }
  leadId = newLead.id;
}

let conversationId: string;
const { data: existingConversation } = await supabaseAdmin
  .from("conversations")
  .select("id")
  .eq("lead_id", leadId)
  .single();

if (existingConversation) {
  conversationId = existingConversation.id;
} else {
  const { data: newConversation, error: conversationError } =
    await supabaseAdmin
      .from("conversations")
      .insert({
        lead_id: leadId,
        org_id: channel.org_id,
        channel_id: channel.id,
        current_state: "new",
      })
      .select("id")
      .single();

  if (conversationError || !newConversation) {
    console.error("đź”´ Failed to create conversation:", conversationError);
    return NextResponse.json(
      { error: "Failed to create conversation" },
      { status: 500 }
    );
  }
  conversationId = newConversation.id;
}

const { data: savedMessage, error: messageError } = await supabaseAdmin
  .from("messages")
  .insert({
    conversation_id: conversationId,
    direction: "inbound",
    subject: nylasMessage.subject || "",
    body_text: plainText,
    body_html: nylasMessage.body || "",
    message_id: nylasMessage.id,
    in_reply_to: nylasMessage.in_reply_to || null,
    raw_payload: nylasMessage,
  })
  .select("id")
  .single();

if (messageError || !savedMessage) {
  console.error("đź”´ Failed to save message:", messageError);
  return NextResponse.json(
    { error: "Failed to save message" },
    { status: 500 }
  );
}

try {
  await processConversationWithChannelConfiguration({ conversationId });
}

6. processConversationWithChannelConfiguration

  1. This is the main workflow for the AI system.
  2. At this point we have determined that this email is related to real estate inqueries, and we have gathered and saved all the relevant data in the database.

7. Load conversation context

Get the conversation database object which is all the messages sent between the sender and the user's email.

If the channel is paused (the user doesn't want the system to respond) then don't do any processing.

export async function processConversationWithChannelConfiguration({
  conversationId,
}: ChannelConfigurationProcessorOptions) {
  try {
    // 1. Load conversation context
    const { data: conversation, error: convError } = await supabaseAdmin
      .from("conversations")
      .select(
        `
        *,
        lead:leads (
          *,
          contact:contacts (*)
        ),
        channel:channels (*),
        messages:messages (
          *
        )
      `
      )
      .eq("id", conversationId)
      .single();

    if (convError || !conversation) {
      console.error(
        `đź”´ Channel Configuration Processor Error: Could not load conversation [${conversationId}]`,
        convError
      );
      return;
    }

    // Additional debugging for channel
    if (!conversation.channel) {
      console.error(
        `đź”´ Conversation ${conversationId} has no channel relationship!`
      );
      console.error(`   Expected channel_id: ${conversation.channel_id}`);

      // Try to find the channel directly
      const { data: directChannel, error: directChannelError } =
        await supabaseAdmin
          .from("channels")
          .select("*")
          .eq("id", conversation.channel_id)
          .single();

      if (directChannelError) {
        console.error(`đź”´ Direct channel lookup failed:`, directChannelError);
      } else {
        console.error(
          `đź”´ Channel ${conversation.channel_id} does not exist in database`
        );
      }
      return;
    }

    // Check if the channel is paused
    if (conversation.channel?.is_paused) {
      return;
    }

    const allMessages = (conversation.messages as any[]) || [];

    // Always process the most recent inbound message
    const lastMessage = allMessages
      .filter((m) => m.direction === "inbound")
      .sort(
        (a, b) =>
          new Date(b.created_at || 0).getTime() -
          new Date(a.created_at || 0).getTime()
      )[0];

    if (!lastMessage) {
      return;
    }

8. Check if the last message was replied to or not

To ensure we aren't sending multiple emails to the same email. We only want to reply once to each email.

// Check if this message has already been replied to
if (lastMessage.replied_to) {
  console.log(
    `âś… Channel Config: Message [${lastMessage.id}] has already been replied to. Skipping processing for conversation [${conversationId}].`
  );
  return;
}

9. Load the channel configuration

Data the user configured about how they want their channel to handle the email operations, such as:

  • Qualifying questions
  • Appointment settings
  • Lead types
  • And more
const { data: channelConfig, error: configError } = await supabaseAdmin
  .from("channel_configurations")
  .select("*")
  .eq("channel_id", conversation.channel_id)
  .single();

if (configError || !channelConfig) {
  return;
}

10. classifyAndExtract

  1. This takes the email text.
  2. Create an AI function tool to classify the intent and entities.
  3. Call Gemini API to analyze the email content and return data about if its related to buying, selling, renting, or investing.
  4. Return the intent, entities, and confidence.
export async function classifyAndExtract(
  text: string
): Promise<ClassificationResult> {
  try {
    // Validate input
    if (!text || typeof text !== "string") {
      throw new Error(
        `Invalid input: text must be a non-empty string, got ${typeof text}`
      );
    }

    // Validate API key
    if (!process.env.GOOGLE_AI_API_KEY) {
      throw new Error("GOOGLE_AI_API_KEY environment variable is not set");
    }

    const classificationToolDeclaration: FunctionDeclaration = {
      name: "classify_email_intent_and_entities",
      parameters: {
        type: Type.OBJECT,
        description:
          "Classifies the intent of an email and extracts key entities.",
        properties: {
          intent: {
            type: Type.STRING,
            enum: [...INTENTS],
            description: "The primary intent of the email.",
          },
          entities: {
            type: Type.ARRAY,
            items: {
              type: Type.OBJECT,
              properties: {
                type: {
                  type: Type.STRING,
                  enum: [...ENTITIES],
                  description:
                    "The type of the extracted entity (e.g., 'date', 'location').",
                },
                value: {
                  type: Type.STRING,
                  description: "The extracted value of the entity.",
                },
              },
              required: ["type", "value"],
            },
            description: "A list of all entities found in the email.",
          },
          confidence: {
            type: Type.NUMBER,
            description:
              "A confidence score from 0.0 to 1.0 about the classification.",
          },
        },
        required: ["intent", "entities", "confidence"],
      },
    };

    const response = await ai.models.generateContent({
      model: "gemini-2.5-flash",
      contents: [
        {
          role: "user",
          parts: [
            {
              text: `You are an expert AI assistant for a real estate business. Analyze this email and classify the user's intent and extract key entities.

IMPORTANT: Only classify as real estate related intents if the email is clearly about real estate services (buying, selling, renting, investing in property). 

- "other" = Non-real estate emails (newsletters, spam, personal emails, business unrelated to real estate)
- "not_interested" = People explicitly saying they don't want to be contacted or are unsubscribing
- "initial_inquiry" = First contact about real estate services
- "price_question" = Questions about property prices, values, or costs
- "showing_request" = Requests to view properties or schedule appointments
- "follow_up" = Follow-up questions about previous real estate discussions

Email to analyze:
${text.slice(0, 4000)}`,
            },
          ],
        },
      ],
      config: {
        toolConfig: {
          functionCallingConfig: {
            mode: FunctionCallingConfigMode.ANY,
            allowedFunctionNames: ["classify_email_intent_and_entities"],
          },
        },
        tools: [{ functionDeclarations: [classificationToolDeclaration] }],
      },
    });

    const functionCalls = response.functionCalls;
    if (!functionCalls || !functionCalls[0]?.args) {
      console.error("đź”´ AI analysis failed to return a valid function call.");
      console.error("Response:", JSON.stringify(response, null, 2));
      return { intent: "other", entities: [], confidence: 0.3 };
    }

    const args = functionCalls[0].args as {
      intent: Intent;
      entities: ExtractedEntity[];
      confidence: number;
    };

    // Validate the response
    if (!args.intent || !INTENTS.includes(args.intent)) {
      console.error("đź”´ AI returned invalid intent:", args.intent);
      return { intent: "other", entities: [], confidence: 0.3 };
    }

    return {
      intent: args.intent || "other",
      entities: args.entities || [],
      confidence: args.confidence || 0.5,
    };
  } catch (error) {
    console.error("đź”´ Error in classifyAndExtract:", error);

    // Provide more context about the error
    if (error instanceof Error) {
      if (error.message.includes("Could not load the default credentials")) {
        console.error(
          "đź”´ Google AI authentication error: Please check GOOGLE_AI_API_KEY environment variable"
        );
      } else if (error.message.includes("API key")) {
        console.error(
          "đź”´ Google AI API key error: Please verify your API key is valid"
        );
      } else if (
        error.message.includes("quota") ||
        error.message.includes("rate limit")
      ) {
        console.error(
          "đź”´ Google AI quota/rate limit error: Please check your usage limits"
        );
      }
    }

    // Return a default error state
    return { intent: "other", entities: [], confidence: 0.0 };
  }
}

11. determineLeadType and classifyLeadType

  1. This takes the classification, email content, and channel configuration.
  2. Calls classifyLeadType.
  3. Creates a function tool to classify the lead type from the email.
  4. Calls Gemini API to analyze the email.
  5. Returns the lead type and confidence.

Now we have the lead type. If the lead type is unclear then we generate a clarification email to send to the user to figure out what type of lead they are.

export async function determineLeadType(
  classification: any,
  emailText: string,
  config: ChannelConfiguration
): Promise<"buyer" | "seller" | "investor" | "renter" | "unclear" | null> {
  // Check which lead types are enabled
  const enabledTypes = Object.entries(config.lead_types_enabled)
    .filter(([_, enabled]) => enabled)
    .map(([type, _]) => type);

  if (enabledTypes.length === 0) {
    return null;
  }

  // Use AI to determine lead type based on email content and classification
  try {
    const leadTypeClassification = await classifyLeadType(
      emailText,
      classification,
      enabledTypes
    );
    return leadTypeClassification;
  } catch (error) {
    console.error("Error classifying lead type with AI:", error);
    // Return unclear instead of falling back to first enabled type
    return "unclear";
  }
}

/**
 * Use AI to classify the lead type based on email content
 */
async function classifyLeadType(
  emailText: string,
  classification: any,
  enabledTypes: string[]
): Promise<"buyer" | "seller" | "investor" | "renter" | "unclear"> {
  try {
    const ai = new GoogleGenAI({ apiKey: process.env.GOOGLE_AI_API_KEY! });

    const leadTypeToolDeclaration: FunctionDeclaration = {
      name: "classify_lead_type",
      parameters: {
        type: Type.OBJECT,
        description: "Classifies the lead type based on email content.",
        properties: {
          lead_type: {
            type: Type.STRING,
            enum: [...enabledTypes, "unclear"],
            description:
              "The determined lead type from the available options, or 'unclear' if cannot be determined with confidence.",
          },
          confidence: {
            type: Type.NUMBER,
            description:
              "Confidence score from 0.0 to 1.0 about the classification.",
          },
        },
        required: ["lead_type", "confidence"],
      },
    };

    const response = await ai.models.generateContent({
      model: "gemini-2.5-flash",
      contents: [
        {
          role: "user",
          parts: [
            {
              text: `Analyze this email and determine the lead type. The available lead types are: ${enabledTypes.join(
                ", "
              )}.

Email content:
${emailText}

Previous AI classification:
- Intent: ${classification.intent}
- Confidence: ${classification.confidence}
- Entities: ${JSON.stringify(classification.entities)}

Based on the email content and context, classify this as one of the following lead types:
${enabledTypes
  .map((type) => {
    switch (type) {
      case "buyer":
        return "- buyer: Someone looking to purchase a property (keywords: buy, purchase, looking to buy, want to buy, interested in buying)";
      case "seller":
        return "- seller: Someone looking to sell their property (keywords: sell, selling, want to sell, need to sell, listing)";
      case "investor":
        return "- investor: Someone looking for investment opportunities (keywords: invest, investment, ROI, rental income, cash flow)";
      case "renter":
        return "- renter: Someone looking to rent a property (keywords: rent, renting, looking to rent, want to rent, lease, apartment, rental)";
      default:
        return `- ${type}: Someone looking for ${type} services`;
    }
  })
  .join("\n")}
- unclear: If the email doesn't clearly indicate the lead type or is ambiguous

IMPORTANT: Look for specific keywords in the email. If someone mentions "rent", "renting", "apartment", "lease", etc., they are likely a renter. If they mention "buy", "purchase", "house", etc., they are likely a buyer. If they mention "sell", "selling", "listing", etc., they are likely a seller.

Choose the most appropriate lead type from the available options. If the email is unclear or doesn't provide enough information to determine the lead type with confidence, return "unclear".`,
            },
          ],
        },
      ],
      config: {
        toolConfig: {
          functionCallingConfig: {
            mode: FunctionCallingConfigMode.ANY,
            allowedFunctionNames: ["classify_lead_type"],
          },
        },
        tools: [{ functionDeclarations: [leadTypeToolDeclaration] }],
      },
    });

    const functionCalls = response.functionCalls;
    if (!functionCalls || !functionCalls[0]?.args) {
      console.error(
        "đź”´ AI lead type classification failed to return a valid function call."
      );
      return "unclear";
    }

    const args = functionCalls[0].args as {
      lead_type: string;
      confidence: number;
    };

    // Validate the response
    if (
      !args.lead_type ||
      (!enabledTypes.includes(args.lead_type) && args.lead_type !== "unclear")
    ) {
      console.error("đź”´ AI returned invalid lead type:", args.lead_type);
      return "unclear";
    }

    // If confidence is low, return unclear
    if (args.confidence < 0.6) {
      return "unclear";
    }

    return args.lead_type as any;
  } catch (error) {
    console.error("Error in AI lead type classification:", error);
    // Return unclear instead of falling back to first enabled type
    return "unclear";
  }
}

13. generateClarificationResponse

  1. Takes the email text, conversation, configuration, and detected lead type.
  2. Checks if the user has a premade email template for clarification emails, otherwise generates one with AI.
  3. Returns the subject and body.
  4. Send the email.
  5. Mark the email as responded to.
  6. End flow.
if (leadType === "unclear") {
  const clarificationResponse = await generateClarificationResponse(
    lastMessage,
    conversation,
    channelConfig
  );

  if (clarificationResponse) {
    const messageId = await sendEmail({
      to: conversation.lead.contact.email,
      subject: clarificationResponse.subject,
      body: clarificationResponse.body,
      from: conversation.channel.identifier,
      channelId: conversation.channel_id,
      inReplyTo: lastMessage.message_id,
    });

    // Save the outbound message with encryption
    const { error: saveError } = await createEncryptedInsert(
      conversation.lead.org_id, // Use org_id as user_id for encryption
      "messages",
      {
        conversation_id: conversationId,
        direction: "outbound",
        subject: clarificationResponse.subject,
        body_text: clarificationResponse.body,
        message_id: messageId,
        in_reply_to: lastMessage.message_id,
      }
    );

    if (saveError) {
      console.error(
        `đź”´ Channel Config: Could not save clarification message for [${conversationId}]:`,
        saveError
      );
    } else {
      // Mark the inbound message as replied to
      const { error: updateError } = await createEncryptedUpdate(
        conversation.lead.org_id, // Use org_id as user_id for encryption
        "messages",
        { replied_to: true },
        { id: lastMessage.id }
      );

      if (updateError) {
        console.error(
          `đź”´ Channel Config: Could not mark message [${lastMessage.id}] as replied to:`,
          updateError
        );
      } else {
        console.log(
          `âś… Channel Config: Marked message [${lastMessage.id}] as replied to for conversation [${conversationId}]`
        );
      }
    }
  }
  return;
}

export async function generateClarificationResponse(
  message: any,
  conversation: any,
  config: ChannelConfiguration,
  detectedLeadType?: string // Add optional parameter for detected but disabled lead type
): Promise<{ subject: string; body: string } | null> {
  try {
    const contactName = conversation.lead?.contact?.name || "there";
    const enabledTypes = Object.entries(config.lead_types_enabled)
      .filter(([_, enabled]) => enabled)
      .map(([type, _]) => type);

    // Use user's qualification template if available
    if (config.communication_settings?.qualificationEmails?.length) {
      const templates = config.communication_settings.qualificationEmails;
      const selectedTemplate =
        templates[Math.floor(Math.random() * templates.length)];

      const emailBody = selectedTemplate
        .replace(/\[Lead Name\]/g, contactName)
        .replace(
          /\[Your Name\]/g,
          config.communication_settings?.emailSignature || "[Your Name]"
        );

      return {
        subject: `${message.subject || "Your inquiry"} - Quick question`,
        body: emailBody,
      };
    }

    // Fallback to AI generation if no templates available
    const context = {
      enabledLeadTypes: enabledTypes,
      enabledLeadTypesList: enabledTypes.join(", "),
      contactName,
      originalMessage: message.body_text,
      communicationSettings: config.communication_settings,
      detectedLeadType,
      instructions: detectedLeadType
        ? `The customer's email appears to be about ${detectedLeadType} services, but we only handle ${enabledTypes.join(
            ", "
          )}. Politely explain that we specialize in ${enabledTypes.join(
            ", "
          )} and ask if they might be interested in those services instead.`
        : `The customer's email is unclear about which real estate service they need. Ask them to specify which of these enabled services they're interested in: ${enabledTypes.join(
            ", "
          )}. Be specific and mention each enabled service type.`,
    };

    const response = await composeReply({
      conversationHistory: message.body_text,
      plan: {
        action: detectedLeadType
          ? `Politely explain we specialize in ${enabledTypes.join(
              ", "
            )} and ask if they're interested in those services instead`
          : `Ask for clarification about which of the enabled lead types they're interested in: ${enabledTypes.join(
              ", "
            )}`,
        reasoning: detectedLeadType
          ? `The customer appears to be looking for ${detectedLeadType} services, but we only handle ${enabledTypes.join(
              ", "
            )}. We should politely redirect them to our available services.`
          : "The email doesn't clearly indicate what type of real estate service they're looking for. We need to ask them to specify which of our enabled services they're interested in to provide the best assistance.",
      },
      agentPersona:
        config.communication_settings?.responseStyle ||
        `You are a helpful and professional real estate agent. ${
          detectedLeadType
            ? `Politely explain that you specialize in ${enabledTypes.join(
                ", "
              )} and ask if they might be interested in those services instead.`
            : `Ask the customer to clarify which of these specific services they're interested in: ${enabledTypes.join(
                ", "
              )}.`
        } Be concise and straightforward in your communication. Keep emails brief and to the point.`,
      knowledge: JSON.stringify(context),
    });

    if (!response) {
      return null;
    }

    // Add email signature if available
    let emailBody = response.body;
    if (config.communication_settings?.emailSignature) {
      emailBody = `${response.body}\n\n${config.communication_settings.emailSignature}`;
    }

    return {
      subject: `${message.subject || "Your inquiry"} - Quick question`,
      body: emailBody,
    };
  } catch (error) {
    console.error("Error generating clarification response:", error);
    return null;
  }
}

14. If lead type is not enabled, send rejection email.

We have determined the lead type as disabled in the channel configuration, so we send a polite rejection email.

  1. Generate rejection email.
  2. Send the email.
  3. Mark the email as replied to.
  4. End flow.
if (!channelConfig.lead_types_enabled[leadType]) {
  // Lead type detected but not enabled - send polite rejection or clarification
  console.log(
    `🔵 Lead type "${leadType}" detected but not enabled for this channel. Sending clarification.`
  );

  const clarificationResponse = await generateClarificationResponse(
    lastMessage,
    conversation,
    channelConfig,
    leadType // Pass the detected lead type
  );

  if (clarificationResponse) {
    const messageId = await sendEmail({
      to: conversation.lead.contact.email,
      subject: clarificationResponse.subject,
      body: clarificationResponse.body,
      from: conversation.channel.identifier,
      channelId: conversation.channel_id,
      inReplyTo: lastMessage.message_id,
    });

    // Save the outbound message with encryption
    const { error: saveError } = await createEncryptedInsert(
      conversation.lead.org_id, // Use org_id as user_id for encryption
      "messages",
      {
        conversation_id: conversationId,
        direction: "outbound",
        subject: clarificationResponse.subject,
        body_text: clarificationResponse.body,
        message_id: messageId,
        in_reply_to: lastMessage.message_id,
      }
    );

    if (saveError) {
      console.error(
        `đź”´ Channel Config: Could not save clarification message for [${conversationId}]:`,
        saveError
      );
    } else {
      // Mark the inbound message as replied to
      const { error: updateError } = await createEncryptedUpdate(
        conversation.lead.org_id, // Use org_id as user_id for encryption
        "messages",
        { replied_to: true },
        { id: lastMessage.id }
      );

      if (updateError) {
        console.error(
          `đź”´ Channel Config: Could not mark message [${lastMessage.id}] as replied to:`,
          updateError
        );
      } else {
        console.log(
          `âś… Channel Config: Marked message [${lastMessage.id}] as replied to for conversation [${conversationId}]`
        );
      }
    }
  }
  return;
}

We have elminated all other possible options in the previous code. At this point, the email sender needs qualifying questions or a meeting link.

  1. Get qualifying questions.
  2. Get context such as today's date.
  3. Call generateResponse.
const qualifyingQuestions = channelConfig.qualifying_questions[leadType] || [];

const contextData = {
  currentDate: new Date().toLocaleDateString("en-US", {
    weekday: "long",
    year: "numeric",
    month: "long",
    day: "numeric",
  }),
  currentTime: new Date().toLocaleTimeString("en-US", {
    hour: "2-digit",
    minute: "2-digit",
    timeZoneName: "short",
  }),
};

const response = await generateResponse(
  lastMessage,
  conversation,
  classification,
  leadType,
  qualifyingQuestions,
  channelConfig,
  contextData
);

16. generateResponse

This takes in the email message, conversation, lead type, qualifying questions, channel configuration, and context data.

Then it calls analyzeConversationContext

export async function generateResponse(
  message: any,
  conversation: any,
  classification: any,
  leadType: string,
  qualifyingQuestions: any[],
  config: ChannelConfiguration,
  contextData?: {
    currentDate: string;
    currentTime: string;
  }
): Promise<{ subject: string; body: string } | null> {
  try {
    // Use AI to analyze conversation context and determine response type
    const conversationContext = await analyzeConversationContext(
      conversation,
      message,
      qualifyingQuestions
    );

    // Create template context for template extraction
    const templateContext: TemplateContext = {
      contactName: conversation.lead?.contact?.name || "there",
      leadType,
      bookingLink: config.appointment_settings?.bookingLink,
      qualifyingQuestions: qualifyingQuestions.map((q) => q.text),
      signature: config.communication_settings?.emailSignature,
    };

    // Extract templates for the response type
    const templates = extractTemplatesForResponseType(
      config,
      conversationContext.responseType,
      templateContext
    );

    // Format templates for AI prompt injection
    const emailTemplates = formatTemplatesForPrompt(templates);

    switch (conversationContext.responseType) {
      case "qualification_questions":
        // Need to ask qualifying questions
        return await generateQualificationQuestionsResponse(
          message,
          conversation,
          leadType,
          qualifyingQuestions,
          config,
          emailTemplates,
          contextData
        );

      case "booking_link":
        // Ready to book - send booking link
        return await generateBookingResponse(
          message,
          conversation,
          leadType,
          config,
          emailTemplates,
          contextData
        );

      case "rejection": {
        // Not qualified - send rejection
        const qualificationResult = await checkQualificationStatus(
          conversation,
          leadType,
          config
        );
        return await generateRejectionResponse(
          message,
          conversation,
          leadType,
          qualificationResult,
          config,
          emailTemplates,
          contextData
        );
      }

      case "none":
        // No response needed (e.g., waiting for customer response)
        console.log("No response needed based on conversation context");
        return null;

      default:
        console.error(
          `Unknown response type: ${conversationContext.responseType}`
        );
        return null;
    }
  } catch (error) {
    console.error("Error generating response:", error);
    return null;
  }
}

17. analyzeConversationContext

  1. Takes the conversation, message, and qualifying questions.
  2. Create a function tool which determines the response type we want to use. Response types can be: reject, qualify, or send a meeting link.
  3. Calls Gemini API to analyze the conversation, the current message, lead type.
  4. Returns the response type, reason, confidence, and next action.
async function analyzeConversationContext(
  conversation: any,
  currentMessage: any,
  qualifyingQuestions: any[]
) {
  const allMessages = conversation.messages || [];
  const conversationHistory = allMessages
    .sort(
      (a: any, b: any) =>
        new Date(a.created_at).getTime() - new Date(b.created_at).getTime()
    )
    .map((m: any) => {
      // Clean the message content to remove quoted text and HTML entities
      const cleanedText =
        m.direction === "outbound"
          ? cleanEmailBody(m.body_text)
          : decodeHtmlEntities(m.body_text);
      return `${m.direction}: ${cleanedText}`;
    })
    .join("\n");

  try {
    const analysisResult = await analyzeConversation({
      conversationHistory,
      currentMessage: currentMessage.body_text,
      qualifyingQuestions,
      leadType: conversation.lead?.lead_type,
    });

    return {
      responseType: analysisResult.responseType,
      reason: analysisResult.reason,
      confidence: analysisResult.confidence,
      nextAction: analysisResult.nextAction,
    };
  } catch (error) {
    console.error("Error analyzing conversation context:", error);
    return {
      responseType: "booking_link",
      reason: "Error in analysis - defaulting to booking link",
      confidence: 0.5,
    };
  }
}

export async function analyzeConversation(
  input: ConversationAnalysisInput
): Promise<ConversationAnalysisResult> {
  const { conversationHistory, currentMessage, qualifyingQuestions, leadType } =
    input;

  try {
    // Validate inputs
    if (!conversationHistory || typeof conversationHistory !== "string") {
      throw new Error(
        `Invalid conversationHistory: must be a non-empty string, got ${typeof conversationHistory}`
      );
    }
    if (!currentMessage || typeof currentMessage !== "string") {
      throw new Error(
        `Invalid currentMessage: must be a non-empty string, got ${typeof currentMessage}`
      );
    }

    // Validate API key
    if (!process.env.GOOGLE_AI_API_KEY) {
      throw new Error("GOOGLE_AI_API_KEY environment variable is not set");
    }

    const analysisToolDeclaration: FunctionDeclaration = {
      name: "analyze_conversation_context",
      parameters: {
        type: Type.OBJECT,
        description:
          "Analyzes conversation context and determines response type.",
        properties: {
          responseType: {
            type: Type.STRING,
            enum: [...RESPONSE_TYPES],
            description:
              "The type of response needed based on conversation analysis.",
          },
          reason: {
            type: Type.STRING,
            description: "The reasoning behind the response type decision.",
          },
          confidence: {
            type: Type.NUMBER,
            description:
              "A confidence score from 0.0 to 1.0 about the analysis.",
          },
          nextAction: {
            type: Type.STRING,
            description: "Optional specific action to take next.",
          },
        },
        required: ["responseType", "reason", "confidence"],
      },
    };

    const response = await ai.models.generateContent({
      model: "gemini-2.5-flash",
      contents: [
        {
          role: "user",
          parts: [
            {
              text: `You are an AI assistant that analyzes real estate conversation context.

Based on the conversation history, determine what type of response is needed.

RESPONSE TYPES:
- "qualification_questions": If we need to ask qualifying questions to determine if the lead is qualified
- "booking_link": If the lead is qualified and ready to book an appointment  
- "rejection": If the lead does not meet our qualification criteria
- "none": If no response is needed (e.g., waiting for customer to respond to our questions)

**Conversation History:**
${conversationHistory}

**Current Message:**
${currentMessage}

**Lead Type:** ${leadType || "Unknown"}

**Qualifying Questions Available:** ${qualifyingQuestions.length} questions

Analyze the conversation context and determine the appropriate response type.`,
            },
          ],
        },
      ],
      config: {
        toolConfig: {
          functionCallingConfig: {
            mode: FunctionCallingConfigMode.ANY,
            allowedFunctionNames: [analysisToolDeclaration.name!],
          },
        },
        tools: [{ functionDeclarations: [analysisToolDeclaration] }],
      },
    });

    const functionCalls = response.functionCalls;
    if (!functionCalls || !functionCalls[0]?.args) {
      throw new Error(
        "AI did not return a valid function call for conversation analysis"
      );
    }

    const args = functionCalls[0].args;

    // Validate the response type
    const responseType = args.responseType as ResponseType;
    if (!RESPONSE_TYPES.includes(responseType)) {
      throw new Error(`Invalid response type returned: ${responseType}`);
    }

    // Validate confidence score
    const confidence = args.confidence as number;
    if (typeof confidence !== "number" || confidence < 0 || confidence > 1) {
      throw new Error(`Invalid confidence score: ${confidence}`);
    }

    return {
      responseType,
      reason: args.reason as string,
      confidence,
      nextAction: args.nextAction as string | undefined,
    };
  } catch (error) {
    console.error("Error in analyzeConversation:", error);

    // Return a safe default
    return {
      responseType: "booking_link",
      reason: "Error in analysis - defaulting to booking link",
      confidence: 0.5,
    };
  }
}

18. Get email templates

At this point we have the type of email we want to send so we need to get the email template.

If its a rejection email then we get the rejection template, if it's a qualifying email, we get the qualifying email etc...

export function extractTemplatesForResponseType(
  config: ChannelConfiguration,
  responseType: string,
  context: TemplateContext
): TemplateMatch[] {
  const templates: TemplateMatch[] = [];
  const { contactName, leadType, bookingLink, qualifyingQuestions, signature } =
    context;

  // Helper function to add templates if they exist
  const addTemplates = (
    templateArray: string[] | undefined,
    category: string,
    type: "custom" | "preMade"
  ) => {
    if (templateArray && templateArray.length > 0) {
      templateArray.forEach((template) => {
        templates.push({
          template: replaceTemplatePlaceholders(template, context),
          type,
          category,
        });
      });
    }
  };

  switch (responseType) {
    case "qualification_questions":
      // Use qualifying questions templates
      addTemplates(
        config.communication_settings?.customTemplates
          ?.qualifyingQuestionsEmails,
        "qualifying_questions",
        "custom"
      );
      addTemplates(
        config.communication_settings?.preMadeTemplates
          ?.qualifyingQuestionsEmails,
        "qualifying_questions",
        "preMade"
      );
      break;

    case "booking_link":
      // Use booking link templates
      addTemplates(
        config.communication_settings?.customTemplates?.bookingLinkEmails,
        "booking_link",
        "custom"
      );
      addTemplates(
        config.communication_settings?.preMadeTemplates?.bookingLinkEmails,
        "booking_link",
        "preMade"
      );
      break;

    case "rejection":
      // Use rejection template
      if (config.communication_settings?.rejectionEmailTemplate) {
        templates.push({
          template: replaceTemplatePlaceholders(
            config.communication_settings.rejectionEmailTemplate,
            context
          ),
          type: "custom",
          category: "rejection",
        });
      }
      break;

    case "initial_inquiry":
      // Use qualification templates for initial inquiries
      addTemplates(
        config.communication_settings?.customTemplates?.qualificationEmails,
        "qualification",
        "custom"
      );
      addTemplates(
        config.communication_settings?.preMadeTemplates?.qualificationEmails,
        "qualification",
        "preMade"
      );
      break;

    default:
      // Use general template emails for other cases
      addTemplates(
        config.communication_settings?.customTemplates?.templateEmails,
        "general",
        "custom"
      );
      addTemplates(
        config.communication_settings?.preMadeTemplates?.templateEmails,
        "general",
        "preMade"
      );
      break;
  }

  return templates;
}

function replaceTemplatePlaceholders(
  template: string,
  context: TemplateContext
): string {
  let processedTemplate = template;

  // Replace basic placeholders
  processedTemplate = processedTemplate.replace(
    /\[Lead Name\]/g,
    context.contactName
  );
  processedTemplate = processedTemplate.replace(
    /\[Your Name\]/g,
    context.signature || "[Your Name]"
  );
  processedTemplate = processedTemplate.replace(
    /\[lead type\]/g,
    context.leadType
  );
  processedTemplate = processedTemplate.replace(
    /\[your specialty\]/g,
    context.leadType
  );

  // Replace booking link
  if (context.bookingLink) {
    processedTemplate = processedTemplate.replace(
      /\[Booking Link\]/g,
      context.bookingLink
    );
  }

  // Replace qualifying questions placeholder
  if (context.qualifyingQuestions && context.qualifyingQuestions.length > 0) {
    const formattedQuestions = context.qualifyingQuestions
      .map((q, index) => `${index + 1}. ${q}`)
      .join("\n");
    processedTemplate = processedTemplate.replace(
      /\[Questions will be automatically inserted based on your qualifying questions configuration\]/g,
      formattedQuestions
    );
  }

  // If the template doesn't contain [Your Name] and we have a signature, add it
  if (!processedTemplate.includes("[Your Name]") && context.signature) {
    processedTemplate = processedTemplate.trim() + "\n\n" + context.signature;
  }

  return processedTemplate;
}

19. Generate email and send email

These are the functions to generate emails based on the response type.

async function generateQualificationQuestionsResponse(
  message: any,
  conversation: any,
  leadType: string,
  qualifyingQuestions: any[],
  config: ChannelConfiguration,
  emailTemplates?: string,
  contextData?: {
    currentDate: string;
    currentTime: string;
  }
): Promise<{ subject: string; body: string } | null> {
  const contactName = conversation.lead?.contact?.name || "there";

  // Get the user's qualifying questions for this lead type
  const leadTypeQuestions =
    config.qualifying_questions[
      leadType as keyof typeof config.qualifying_questions
    ] || [];

  // Format questions for email
  const formattedQuestions = leadTypeQuestions
    .map((q, index) => `${index + 1}. ${q.text}`)
    .join("\n");

  // Use user's template if available, otherwise fall back to AI generation
  if (config.communication_settings?.qualifyingQuestionsEmails?.length) {
    const templates = config.communication_settings.qualifyingQuestionsEmails;
    const selectedTemplate =
      templates[Math.floor(Math.random() * templates.length)];

    // Use the new template extraction system for signature handling
    const templateContext: TemplateContext = {
      contactName,
      leadType,
      qualifyingQuestions: leadTypeQuestions.map((q) => q.text),
      signature: config.communication_settings?.emailSignature,
    };

    const extractedTemplates = extractTemplatesForResponseType(
      config,
      "qualification_questions",
      templateContext
    );

    if (extractedTemplates.length > 0) {
      const bestTemplate = getBestTemplateMatch(
        config,
        "qualification_questions",
        templateContext
      );

      if (bestTemplate) {
        return {
          subject: `${message.subject || "Your inquiry"} - Quick questions`,
          body: bestTemplate,
        };
      }
    }

    // Fallback to old method if template extraction fails
    let emailBody = selectedTemplate
      .replace(/\[Lead Name\]/g, contactName)
      .replace(
        /\[Your Name\]/g,
        config.communication_settings?.emailSignature || "[Your Name]"
      );

    emailBody = emailBody.replace(
      /\[Questions will be automatically inserted based on your qualifying questions configuration\]/g,
      formattedQuestions
    );

    return {
      subject: `${message.subject || "Your inquiry"} - Quick questions`,
      body: emailBody,
    };
  }

  // Fallback to AI generation with templates
  const context = {
    leadType,
    qualifyingQuestions: leadTypeQuestions,
    communicationSettings: config.communication_settings,
    contactName,
    originalMessage: message.body_text,
  };

  const response = await composeReply({
    conversationHistory: message.body_text,
    plan: {
      action: `Ask qualifying questions for ${leadType} lead`,
      reasoning: `Customer is a ${leadType} lead but needs to be qualified before proceeding. Ask the qualifying questions to determine if they meet our criteria.`,
    },
    agentPersona:
      config.communication_settings?.responseStyle ||
      `You are a helpful and professional real estate agent. Ask the qualifying questions to determine if this lead meets your criteria. Be concise and straightforward in your communication. Keep emails brief and to the point.`,
    knowledge: JSON.stringify({
      ...context,
      contextData: contextData || generateContextData(),
    }),
    emailTemplates,
  });

  if (!response) {
    return null;
  }

  return {
    subject: `${message.subject || "Your inquiry"} - Quick questions`,
    body: response.body,
  };
}

async function generateBookingResponse(
  message: any,
  conversation: any,
  leadType: string,
  config: ChannelConfiguration,
  emailTemplates?: string,
  contextData?: {
    currentDate: string;
    currentTime: string;
  }
): Promise<{ subject: string; body: string } | null> {
  const contactName = conversation.lead?.contact?.name || "there";
  const bookingLink =
    config.appointment_settings?.bookingLink || "your preferred booking method";

  // Use user's template if available, otherwise fall back to AI generation
  if (config.communication_settings?.bookingLinkEmails?.length) {
    const templates = config.communication_settings.bookingLinkEmails;
    const selectedTemplate =
      templates[Math.floor(Math.random() * templates.length)];

    // Use the new template extraction system for signature handling
    const templateContext: TemplateContext = {
      contactName,
      leadType,
      bookingLink,
      signature: config.communication_settings?.emailSignature,
    };

    const extractedTemplates = extractTemplatesForResponseType(
      config,
      "booking_link",
      templateContext
    );

    if (extractedTemplates.length > 0) {
      const bestTemplate = getBestTemplateMatch(
        config,
        "booking_link",
        templateContext
      );

      if (bestTemplate) {
        return {
          subject: `${message.subject || "Your inquiry"} - Next steps`,
          body: bestTemplate,
        };
      }
    }

    // Fallback to old method if template extraction fails
    let emailBody = selectedTemplate
      .replace(/\[Lead Name\]/g, contactName)
      .replace(
        /\[Your Name\]/g,
        config.communication_settings?.emailSignature || "[Your Name]"
      );

    emailBody = emailBody
      .replace(/\[lead type\]/g, leadType)
      .replace(/\[Booking Link\]/g, bookingLink);

    return {
      subject: `${message.subject || "Your inquiry"} - Next steps`,
      body: emailBody,
    };
  }

  // Fallback to AI generation with templates
  const context = {
    leadType,
    communicationSettings: config.communication_settings,
    appointmentSettings: config.appointment_settings,
    contactName,
    originalMessage: message.body_text,
  };

  const response = await composeReply({
    conversationHistory: message.body_text,
    plan: {
      action: `Provide booking link for qualified ${leadType} lead`,
      reasoning: `Customer is a qualified ${leadType} lead. Provide them with booking information and next steps.`,
    },
    agentPersona:
      config.communication_settings?.responseStyle ||
      `You are a helpful and professional real estate agent. Provide booking information to this qualified lead. Be concise and straightforward in your communication. Keep emails brief and to the point.`,
    knowledge: JSON.stringify({
      ...context,
      contextData: contextData || generateContextData(),
    }),
    emailTemplates,
  });

  if (!response) {
    return null;
  }

  return {
    subject: `${message.subject || "Your inquiry"} - Next steps`,
    body: response.body,
  };
}

async function generateRejectionResponse(
  message: any,
  conversation: any,
  leadType: string,
  qualificationResult: QualificationResult,
  config: ChannelConfiguration,
  emailTemplates?: string,
  contextData?: {
    currentDate: string;
    currentTime: string;
  }
): Promise<{ subject: string; body: string } | null> {
  const contactName = conversation.lead?.contact?.name || "there";

  // Use user's rejection template if available
  if (config.communication_settings?.rejectionEmailTemplate) {
    // Use the new template extraction system for signature handling
    const templateContext: TemplateContext = {
      contactName,
      leadType,
      signature: config.communication_settings?.emailSignature,
    };

    const extractedTemplates = extractTemplatesForResponseType(
      config,
      "rejection",
      templateContext
    );

    if (extractedTemplates.length > 0) {
      const bestTemplate = getBestTemplateMatch(
        config,
        "rejection",
        templateContext
      );

      if (bestTemplate) {
        return {
          subject: message.subject || "Your inquiry",
          body: bestTemplate,
        };
      }
    }

    // Fallback to old method if template extraction fails
    let emailBody = config.communication_settings.rejectionEmailTemplate
      .replace(/\[Lead Name\]/g, contactName)
      .replace(
        /\[Your Name\]/g,
        config.communication_settings?.emailSignature || "[Your Name]"
      );

    emailBody = emailBody.replace(/\[your specialty\]/g, leadType);

    return {
      subject: message.subject || "Your inquiry",
      body: emailBody,
    };
  }

  // Fallback to AI generation with templates
  const context = {
    leadType,
    qualificationResult,
    communicationSettings: config.communication_settings,
    contactName,
    originalMessage: message.body_text,
  };

  const response = await composeReply({
    conversationHistory: message.body_text,
    plan: {
      action: `Politely decline unqualified ${leadType} lead`,
      reasoning: `Customer is a ${leadType} lead but doesn't meet our qualification criteria. Politely explain why we can't help and suggest alternatives.`,
    },
    agentPersona:
      config.communication_settings?.responseStyle ||
      `You are a helpful and professional real estate agent. Politely explain why this lead doesn't meet your criteria and suggest alternatives. Be concise and straightforward in your communication. Keep emails brief and to the point.`,
    knowledge: JSON.stringify({
      ...context,
      contextData: contextData || generateContextData(),
    }),
    emailTemplates,
  });

  if (!response) {
    return null;
  }

  return {
    subject: message.subject || "Your inquiry",
    body: response.body,
  };
}

async function checkQualificationStatus(
  conversation: any,
  leadType: string,
  config: ChannelConfiguration
): Promise<QualificationResult> {
  // Get the qualifying questions for this lead type
  const qualifyingQuestions =
    config.qualifying_questions[
      leadType as keyof typeof config.qualifying_questions
    ] || [];

  if (qualifyingQuestions.length === 0) {
    // No qualifying questions configured - default to qualified
    return {
      qualified: true,
      score: 100,
      evaluations: [],
      strategy: config.qualification_settings.strategy,
      reason: "No qualifying questions configured - default to qualified",
    };
  }

  // Extract lead responses from conversation messages
  const leadResponses: { [questionId: string]: string } = {};
  const allMessages = conversation.messages || [];

  // Look for responses to qualifying questions in the conversation
  for (const message of allMessages) {
    if (message.direction === "inbound") {
      // This is a simplified approach - in a real implementation,
      // you'd want to parse the message to extract specific question responses
      // For now, we'll use the entire message as a response
      const questionId = `question_${qualifyingQuestions.findIndex((q) =>
        message.body_text?.toLowerCase().includes(q.text.toLowerCase())
      )}`;

      if (questionId !== "question_-1") {
        leadResponses[questionId] = message.body_text || "";
      }
    }
  }

  // If no responses found, consider unqualified
  if (Object.keys(leadResponses).length === 0) {
    return {
      qualified: false,
      score: 0,
      evaluations: qualifyingQuestions.map((q) => ({
        questionId: q.id || `question_${qualifyingQuestions.indexOf(q)}`,
        passed: false,
        reason: "no_criteria" as const,
      })),
      strategy: config.qualification_settings.strategy,
      reason: "No responses to qualifying questions found",
    };
  }

  // Use the qualification engine to evaluate the lead
  return evaluateLead(
    leadResponses,
    qualifyingQuestions,
    config.qualification_settings
  );
}

20. Send the email

Lastly we send the generated email and end the workflow.

// 8. Send the response
const messageId = await sendEmail({
  to: conversation.lead.contact.email,
  subject: response.subject,
  body: response.body,
  from: conversation.channel.identifier,
  channelId: conversation.channel_id,
  inReplyTo: lastMessage.message_id,
});

/**
 * Send email using Nylas v3 API
*/
export async function sendEmail(
  grantId: string,
  to: string[],
  subject: string,
  body: string,
  from?: string,
  inReplyTo?: string
): Promise<any> {
  const apiKey = process.env.NYLAS_API_KEY;
  const apiUri = process.env.NYLAS_API_URI || "https://api.us.nylas.com";

  if (!apiKey) {
    throw new Error("NYLAS_API_KEY environment variable is required");
  }

  // Ensure proper line breaks are preserved in the email body
  const formattedBody = body.trim();

  // Convert plain text to HTML with better spacing control
  const lines = formattedBody.split("\n");
  const htmlLines: string[] = [];

  for (let i = 0; i < lines.length; i++) {
    const line= lines[i].trim();

    if (line.length > 0) {
      // Regular content line
      htmlLines.push(`<p styl="margin: 0 0 8px 0;">${line}</p>`);
    } else if (i < lines.length - 1 && lines[i + 1].trim().length > 0) {
      // Empty line followed by content - add a small break
      htmlLines.push('<div styl="height: 4px;"></div>');
    }
    // Skip consecutive empty lines to avoid excessive spacing
  }

  const htmlBody = htmlLines.join("");

  // Enhance email body with better formatting and proper line breaks
  const enhancedHtmlBody = `${htmlBody}

<p styl="margin: 16px 0 8px 0; border-top: 1px solid #e0e0e0; padding-top: 8px;">---</p>
<p styl="margin: 0; font-size: 12px; color: #666;">This email was sent via LeadIntake.ai</p>`;

  const requestBody: any = {
    to: to.map((email) => ({ email })),
    subject: subject,
    body: enhancedHtmlBody,
    body_type: "html", // Specify HTML format
    from: from ? [{ email: from }] : undefined,
    // Add headers to improve deliverability
    headers: {
      "X-Mailer": "LeadIntake.ai",
      "X-Priority": "3",
      "X-MSMail-Priority": "Normal",
      Importance: "Normal",
    },
  };

  // Add reply-to threading if provided
  if (inReplyTo) {
    requestBody.in_reply_to = inReplyTo;
    console.log(`đź“§ Adding reply-to threading: ${inReplyTo}`);
  }

  const response = await fetch(`${apiUri}/v3/grants/${grantId}/messages/send`, {
    method: "POST",
    headers: {
      Authorization: `Bearer ${apiKey}`,
      "Content-Type": "application/json",
    },
    body: JSON.stringify(requestBody),
  });

  if (!response.ok) {
    const error = await response.text();
    throw new Error(`Failed to send email: ${error}`);
  }

  return response.json();
}

Lessons learned

Perhaps too much manual system control.

I think it would be better to give the AI inputs and let it choose a series of possible functions to take, rather than direct it through a specific workflow.

An old school software system would operate based on a -> b -> c etc… but there was no intelligence behind the decision making, just logical steps.

With AI though, you are losing out on a lot of the benefits of AI by designing a system where you explicitly tell it what to do, from a -> b -> c etc…

You should instead create many functionalities and tell AI to analyze the context of the conversation to choose which functionality to do.

I manually set up the system flow and the AI would just do the functionality based on what step in the sequence the flow was operating in.