# Example Scraping Configurations for Wine & Agriculture Websites

## Configuration 1: 8wines.fr Blog
toc:
  url: https://8wines.fr/blog
  _scrap:
    - name: blog_posts
      collection:
        selector: article, .post, .blog-post
        _options:
          post:
            - name: HashValue
        row:
          - name: link
            selector: a
            attr: href
          - name: title
            selector: h1, h2, h3, .title, .post-title
            attr: textContent
            _options:
              post:
                - name: CleanValue
                  params:
                    regexp: '
                    newValue: "'"
                - name: CleanValue
                  params:
                    regexp: –
                    newValue: "-"
          - name: excerpt
            selector: .excerpt, .summary, p
            attr: textContent
          - name: date
            selector: time, .date, .published
            attr: textContent

content:
  _scrap:
    - name: site_title
      selector: h1, .site-title
      attr: textContent
    - name: description
      selector: meta[name="description"]
      attr: content

---

## Configuration 2: Agreste Agriculture (France)
toc:
  url: https://agreste.agriculture.gouv.fr/agreste-web/
  _scrap:
    - name: publications
      collection:
        selector: .publication-item, .document, article
        row:
          - name: title
            selector: h2, h3, .title
            attr: textContent
          - name: link
            selector: a
            attr: href
          - name: date
            selector: .date, time
            attr: textContent
          - name: category
            selector: .category, .type
            attr: textContent
          - name: file_type
            selector: .format, .file-type
            attr: textContent

content:
  _scrap:
    - name: page_title
      selector: h1
      attr: textContent
    - name: main_content
      selector: main, .content, #content
      attr: textContent

---

## Configuration 3: Agriculture Canada Search Results
toc:
  url: https://www.canada.ca/en/agriculture-agri-food/search/advanced-search.html
  _scrap:
    - name: search_results
      collection:
        selector: .srch-rslt, .search-result, article
        _options:
          post:
            - name: HashValue
        row:
          - name: title
            selector: h3, h4, .title
            attr: textContent
          - name: url
            selector: a
            attr: href
          - name: description
            selector: p, .description
            attr: textContent
          - name: date
            selector: time, .date
            attr: textContent

content:
  _scrap:
    - name: results_count
      selector: .result-count, .search-count
      attr: textContent
    - name: search_term
      selector: input[type="search"]
      attr: value

---

## Configuration 4: Australian Agriculture (ABARES)
toc:
  url: https://www.agriculture.gov.au/abares
  _scrap:
    - name: resources
      collection:
        selector: .resource, .publication, article
        row:
          - name: title
            selector: h2, h3
            attr: textContent
            _options:
              post:
                - name: CleanValue
                  params:
                    regexp: '\s+'
                    newValue: ' '
          - name: link
            selector: a
            attr: href
          - name: description
            selector: p, .summary
            attr: textContent
          - name: date
            selector: time, .date
            attr: textContent

content:
  _scrap:
    - name: page_heading
      selector: h1
      attr: textContent
    - name: breadcrumb
      selector: .breadcrumb
      attr: textContent

---

## Configuration 5: ACCC Australia Search
toc:
  url: https://www.accc.gov.au/search
  _scrap:
    - name: search_results
      collection:
        selector: .search-result, .result-item
        row:
          - name: title
            selector: h3, .result-title
            attr: textContent
          - name: url
            selector: a
            attr: href
          - name: snippet
            selector: .snippet, .description
            attr: textContent
          - name: date
            selector: .date, time
            attr: textContent

content:
  _scrap:
    - name: total_results
      selector: .results-count
      attr: textContent

---

## Configuration 6: Austrian Wine
toc:
  url: https://www.austrianwine.com/
  _scrap:
    - name: news_items
      collection:
        selector: .news-item, article, .post
        row:
          - name: headline
            selector: h2, h3, .headline
            attr: textContent
          - name: link
            selector: a
            attr: href
          - name: image
            selector: img
            attr: src
          - name: summary
            selector: p, .summary
            attr: textContent

content:
  _scrap:
    - name: main_headline
      selector: h1
      attr: textContent
    - name: featured_content
      selector: .featured, .hero
      attr: textContent

---

## Configuration 7: Austrian Wine Brochures
toc:
  url: https://www.austrianwine.com/press-media/brochures/brochures
  _scrap:
    - name: brochures
      collection:
        selector: .brochure, .download-item, .document
        _options:
          post:
            - name: HashValue
        row:
          - name: title
            selector: h3, h4, .title
            attr: textContent
            _options:
              post:
                - name: CleanValue
                  params:
                    regexp: '"'
                    newValue: "'"
          - name: download_link
            selector: a
            attr: href
          - name: file_size
            selector: .size, .file-size
            attr: textContent
          - name: format
            selector: .format, .type
            attr: textContent
          - name: description
            selector: p, .description
            attr: textContent

content:
  _scrap:
    - name: page_title
      selector: h1
      attr: textContent
    - name: category
      selector: .breadcrumb a:last-child
      attr: textContent

---

## Universal Configuration (Works on Most Sites)
toc:
  url: https://example.com
  _scrap:
    - name: main_items
      collection:
        selector: article, .item, .post, .product, .card
        _options:
          post:
            - name: HashValue
        row:
          - name: title
            selector: h1, h2, h3, .title, .heading
            attr: textContent
          - name: link
            selector: a
            attr: href
          - name: description
            selector: p, .description, .excerpt, .summary
            attr: textContent
          - name: image
            selector: img
            attr: src
          - name: date
            selector: time, .date, .published
            attr: textContent

content:
  _scrap:
    - name: page_title
      selector: h1
      attr: textContent
    - name: meta_description
      selector: meta[name="description"]
      attr: content
    - name: main_content
      selector: main, .main-content, #content
      attr: textContent

---

## Tips for Customizing These Configurations:

1. **Inspect the Target Website**: 
   - Open browser DevTools (F12)
   - Find the elements you want to scrape
   - Copy their CSS selectors or create XPath

2. **Test Selectors**:
   - In browser console: `document.querySelectorAll('.your-selector')`
   - For XPath: `$x('//your/xpath')`

3. **Adjust Selectors**:
   - Replace generic selectors with site-specific ones
   - Use commas to provide fallback selectors: `h1, .title, .heading`

4. **Add More Fields**:
   ```yaml
   - name: new_field
     selector: .your-selector
     attr: textContent
   ```

5. **Chain Post-Processors**:
   ```yaml
   _options:
     post:
       - name: CleanValue
         params:
           regexp: '\s+'
           newValue: ' '
       - name: HashValue
   ```

## Common Selector Patterns:

### Articles/Blog Posts
```
article, .post, .blog-post, .article-item
h2, h3, .post-title, .article-title
.post-content, .article-body
```

### Search Results
```
.search-result, .result-item, .srch-rslt
.result-title, .search-title
.snippet, .description
```

### Products/Items
```
.product, .item, .card, .listing
.product-name, .item-title
.price, .product-price
```

### Navigation/Links
```
nav a, .menu-item, .nav-link
.breadcrumb, .breadcrumb-item
```

### Metadata
```
meta[name="description"]
meta[property="og:title"]
time, .date, .published
```
