Playwright results in an empty array when going to another page during a loop

  Kiến thức lập trình

I am trying to parse a page with Playwright.

The page has a table and each row contains data including a URL that I want to have Playwright goto, grab extra information and then add it to the row object that I am saving.

It works fine until I add the code to visit the newPage in which case I get an empty array.

I have a feeling it has to do with using await within a map function? Can someone offer some guidance?

const browserType = "chromium";
const url = "https://page-with-table.html";
const browser = await playwright[browserType].launch();
const context = await browser.newContext();
const page = await context.newPage();

await page.goto(url);

const tableRowsLocator = page.locator("table tr");

const tableData = await tableRowsLocator.evaluateAll(async (tableRows) => {
  const data = [];

  tableRows.map(async (tableRow) => {
    const rowObject = {};

    const rowData = tableRow.querySelector('td[data="rowData"]');

    const url = tableRow.querySelector('td[data="url"] a');

    // --- Problem code begins

    const newPage = await context.newPage();

    await newPage.goto(url);

    const contentLocator = newPage.locator("#content");

    const contentData = await contentLocator.evaluate(async (content) => {
      return content.textContent;
    });

    rowObject.content = newPageData;

    await newPageData.waitForTimeout(1000);
    await newPageData.close();

    // --- Problem code ends

    rowObject.rowData = rowData.textContent;
    rowObject.url = url.getAttribute("href");

    data.push(rowObject);
  });

  return data;
});

await page.waitForTimeout(1000);
await browser.close();

console.log(gamesData);

LEAVE A COMMENT