{"article":{"id":38203604451085,"url":"https://bardeensupport.zendesk.com/api/v2/help_center/en-us/articles/38203604451085.json","html_url":"https://support.bardeen.ai/hc/en-us/articles/38203604451085-How-to-Configure-a-Browser-Agent-to-extract-data","author_id":20092547360269,"comments_disabled":false,"draft":false,"promoted":true,"position":0,"vote_sum":0,"vote_count":0,"section_id":23925234876173,"created_at":"2025-07-21T11:29:11Z","updated_at":"2025-08-01T12:34:35Z","name":"How to Configure a Browser Agent to extract data","title":"How to Configure a Browser Agent to extract data","source_locale":"en-us","locale":"en-us","outdated":false,"outdated_locales":[],"edited_at":"2025-08-01T12:34:35Z","user_segment_id":null,"permission_group_id":20101069543181,"content_tag_ids":["01JFEZHP985GJYYKNJZADM6VJZ","01K0PBWW80218CJY469RR8FJR8"],"label_names":[],"body":"<p><span class=\"wysiwyg-font-size-large\">Bardeen’s Browser Agent Builder allows you to configure an <strong>agent</strong> that can extract information from a webpage. It’s fast and intuitive: you select elements on an example page, and Bardeen will repeat the logic on similar pages.</span></p><p><span class=\"wysiwyg-font-size-large\">You can configure agents to extract:</span></p><ul>\n<li>\n<span class=\"wysiwyg-font-size-large\">Specific fields from a </span><a href=\"#single-page-agent\"><span class=\"wysiwyg-font-size-large\"><strong>single page</strong></span></a>\n</li>\n<li>\n<span class=\"wysiwyg-font-size-large\">Repeating information from </span><a href=\"#list-extraction-agent\"><span class=\"wysiwyg-font-size-large\"><strong>lists</strong></span></a>\n</li>\n<li>\n<span class=\"wysiwyg-font-size-large\">Structured </span><a href=\"#table-extraction-agent\"><span class=\"wysiwyg-font-size-large\"><strong>tables</strong></span></a>\n</li>\n<li>\n<span class=\"wysiwyg-font-size-large\">Or even <strong>deep crawl</strong> by opening sub-pages (see </span><a href=\"https://support.bardeen.ai/hc/en-us/articles/38447790401549\"><span class=\"wysiwyg-font-size-large\">Deep Crawling with Agents</span></a><span class=\"wysiwyg-font-size-large\">)</span>\n</li>\n<li>\n<span class=\"wysiwyg-font-size-large\">You can also build agents that extract <strong>multiple lists from one page</strong>. Learn more in </span><a href=\"#\"><span class=\"wysiwyg-font-size-large\">How to configure an agent for extracting multiple lists?</span></a><span class=\"wysiwyg-font-size-large\"> </span>\n</li>\n</ul><p> </p><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"single-page-agent\">📄 Single Page Extraction</h2><p><span class=\"wysiwyg-font-size-large\">A single page agent extracts static elements like text, links, or images from into a single row of data.</span></p><h3 id=\"h_01K0PAPTS82CWAW7WQKG6WME5X\"><span class=\"wysiwyg-font-size-large\">Start Building</span></h3><ol>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">In the <strong>Scrape data on active tab (or in the background)</strong> action, click <strong>+ Create Scraper</strong>.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Select or paste a link to an example page.</span>\n    </li>\n  </ol><p id=\"h_note\" style=\"background-color: #fef3c7; border: 1px solid #facc15; border-radius: 10px; padding: 1em;\">\n    <span class=\"wysiwyg-font-size-large\"><strong>⚠️ Important:</strong><br> This URL becomes the template. You can't change it later without rebuilding, so choose a page that contains <strong>all data fields</strong> you want to extract. </span>\n  </p><figure style=\"width: 53.64%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 519/293;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203604445965\" alt=\"tmunp.png\" width=\"519\" height=\"293\"></figure><h3 id=\"h_01K0PAPTSAWY1ZD0ZZD9ZAB8C4\"><span class=\"wysiwyg-font-size-large\">Select Elements</span></h3><ol>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Click on highlighted elements and choose <strong>Get text</strong>, <strong>Get link</strong>, etc.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Name the field (e.g., “Company Name”, “Industry”).</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Repeat for each item.</span>\n    </li>\n  </ol><p> </p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 973/709;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203589779213\" alt=\"zaby9.png\" width=\"973\" height=\"709\"></figure><p> </p><hr style=\"border: 0.5px solid #DDDDDD;\"><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"advanced-actions\">🖱️ Advanced Actions</h2><p><span class=\"wysiwyg-font-size-large\">Depending on how the page behaves, you may need to interact with it to extract information.</span></p><ul>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Click</strong> – Expand or trigger elements.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Input</strong> – Enter values or fill forms.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Get Image</strong> – Extract image URLs.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Hover</strong> – Reveal hover-only content.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Get HTML</strong> – Capture raw HTML of an element.</span></li>\n</ul><figure style=\"width: 35.84%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 492/770;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203604447501\" alt=\"49njw.png\" width=\"492\" height=\"770\"></figure><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><p><span class=\"wysiwyg-font-size-large\">To remove or rename a field, use the menu on the right side. You can also add extra logic using the <strong>+</strong> button (see </span><a href=\"#special-elements\"><span class=\"wysiwyg-font-size-large\">Special Elements</span></a><span class=\"wysiwyg-font-size-large\">).</span></p><p> </p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 1396/740;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203668989069\" alt=\"q2sfo.png\" width=\"1396\" height=\"740\"></figure><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"special-elements\">➕ Special Operation</h2><p><span class=\"wysiwyg-font-size-large\">To refine your agent logic, you can use additional actions by clicking the <strong>➕</strong> button in the Builder, those are the actions that doesn't require specific element on a page.</span></p><ul>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Delay</strong> – Wait before the next action. Useful after a <em>Click</em>.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Meta image / title / description</strong> – Extract page-level metadata.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Page URL</strong> – Add the current page’s link.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Clipboard</strong> – Access copied content.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Time scraped</strong> – Timestamp of the run.</span></li>\n</ul><figure style=\"width: 66.22%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 795/778;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203604446349\" alt=\"sn6ei.png\" width=\"795\" height=\"778\"></figure><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"list-extraction-agent\">📋 List Extraction</h2><p><span class=\"wysiwyg-font-size-large\">List operations allow extracting data from repeating sections like product listings or search results from a website.</span></p><h3 id=\"h_01K0PAPTSA87F59Y3F1AWHFXHW\"><span class=\"wysiwyg-font-size-large\">Create the List Logic</span></h3><ol>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Open the page with the list.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Click on a single item (e.g., one result).</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Choose <strong>List</strong> to enter list mode.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Click on another similar item to define the structure.</span>\n    </li>\n  </ol><p id=\"h_note\" style=\"background-color: #fef3c7; border: 1px solid #facc15; border-radius: 10px; padding: 1em;\">\n    <span class=\"wysiwyg-font-size-large\"><strong>Note:</strong><br> You only need to define the structure with two examples — Bardeen will repeat it for the rest. </span>\n  </p><p> </p><figure style=\"width: 50.68%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 513/482;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203589780621\" alt=\"eb8jm.png\" width=\"513\" height=\"482\"></figure><h3 id=\"h_01K0PAPTSAHYCAJCHGP7N586Y8\"><span class=\"wysiwyg-font-size-large\">Configure Pagination</span></h3><p><span class=\"wysiwyg-font-size-large\">If there are multiple pages:</span></p><ul>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Click pagination</strong> – Pick a “Next” or “Show more” button.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Infinite scroll</strong> – Auto-loads on scroll.</span></li>\n</ul><figure style=\"width: 48%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 394/338;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203589781005\" alt=\"iknms.png\" width=\"394\" height=\"338\"></figure><p>After current step you should see all the relevant items highlighted:</p><p> </p><figure style=\"width: 97.49%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 3534/1830;\" src=\"https://support.bardeen.ai/hc/article_attachments/38446574977677\" width=\"3534\" height=\"1830\"></figure><p>Now you can start picking items inside of the list, similar to how you would do in <a href=\"#single-page-agent\">Single Page Extraction</a>:</p><figure style=\"width: 68.8%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 1108/1028;\" src=\"https://support.bardeen.ai/hc/article_attachments/38446574980237\" width=\"1108\" height=\"1028\"></figure><p>But now this action will be repeated for every single list item and extract every single item from the defined list:</p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 3536/1818;\" src=\"https://support.bardeen.ai/hc/article_attachments/38446589282701\" width=\"3536\" height=\"1818\"></figure><p> </p><p> </p><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"table-extraction-agent\">🧾 Table Extraction Agent</h2><p><span class=\"wysiwyg-font-size-large\">If a webpage contains a table, Bardeen can auto-detect and extract it with structure.</span></p><ol>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Click any <strong>cell</strong> inside a table.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">Choose <strong>Table</strong> from the menu.</span>\n    </li>\n    <li>\n      <span class=\"wysiwyg-font-size-large\">To extract additional elements like links, click <strong>Activate</strong> and then select more fields.</span>\n    </li>\n  </ol><p> </p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 1818/1194;\" src=\"https://support.bardeen.ai/hc/article_attachments/38446574985485\" width=\"1818\" height=\"1194\"></figure><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 859/893;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203604448141\" alt=\"cr7uo.png\" width=\"859\" height=\"893\"></figure><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"h_01K1EAGMN7W9YZAYQZFFN9C40K\">🧾 Editing Browser Agent Template with a List</h2><p>When editing agent model and adding new fields to existing list item like in the image below:</p><p> </p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 3502/1848;\" src=\"https://support.bardeen.ai/hc/article_attachments/38447430369549\" width=\"3502\" height=\"1848\"></figure><p>You might notice that the item in certain situation is being added as a separate field:</p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 3538/1832;\" src=\"https://support.bardeen.ai/hc/article_attachments/38447444339085\" width=\"3538\" height=\"1832\"></figure><p>This is because the lists are disabled by default and every item added is treated as <a href=\"#single-page-agent\">single page item</a> so you can combine both single and list operations, but if intend to add item to the list then we need to activate it first by clicking on activate icon:</p><figure class=\"wysiwyg-image\"><img style=\"aspect-ratio: 3506/1832;\" src=\"https://support.bardeen.ai/hc/article_attachments/38447430373517\" width=\"3506\" height=\"1832\"></figure><p> </p><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"agent-settings\">⚙️ Agent Settings</h2><p><span class=\"wysiwyg-font-size-large\">Click the <strong>gear icon</strong> in the Builder to configure options.</span></p><h3 id=\"h_01K0PAPTSBNN4YZNEJ3XSZ14N5\"><span class=\"wysiwyg-font-size-large\">Where should the agent run?</span></h3><ul>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Everywhere</strong> – Use globally.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>On current domain</strong> – Recommended default.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>Custom</strong> – Use custom logic or regex.</span></li>\n</ul><h3 id=\"h_01K0PAPTSB184VW1HWGM06N395\"><span class=\"wysiwyg-font-size-large\">Other settings:</span></h3><ul>\n<li><span class=\"wysiwyg-font-size-large\"><strong>✅ Use minimized window</strong> – Turn off for sites like LinkedIn that block minimized windows.</span></li>\n<li><span class=\"wysiwyg-font-size-large\"><strong>✅ Scrape in the cloud</strong> – Works only on <strong>public pages</strong>. Not supported for gated sites.</span></li>\n</ul><figure style=\"width: 86.43%;\" class=\"wysiwyg-image wysiwyg-image-resized\"><img style=\"aspect-ratio: 927/576;\" src=\"https://support.bardeen.ai/hc/article_attachments/38203604448525\" alt=\"scohy.png\" width=\"927\" height=\"576\"></figure><hr style=\"border: 0.5px solid #DDDDDD;\"><p> </p><h2 style=\"color: #6f60cd; font-weight: bolder;\" id=\"h_01K0PAPTSBMTMH5TJ4SM6BND5Z\">👇 What’s Next?</h2><ul>\n<li>\n<a href=\"https://support.bardeen.ai/hc/en-us/articles/38447790401549\"><span class=\"wysiwyg-font-size-large\">🔍 How to Configure a Deep Crawling Agent</span></a><span class=\"wysiwyg-font-size-large\"> – Navigate into subpages and extract more layers of data.</span>\n</li>\n<li><a href=\"#\"><span class=\"wysiwyg-font-size-large\">🧩 How to configure an agent for extracting multiple lists?</span></a></li>\n</ul>"}}