Skip to content

README.md

Latest commit

 

History

History
124 lines (95 loc) · 3.63 KB

README.md

File metadata and controls

124 lines (95 loc) · 3.63 KB
Raw

ScrapeScribe Frontend

A modern web interface for interactive web scraping and API generation. This frontend application allows users to visually select elements from web pages and generate custom APIs for data extraction.

🚀 Features

  • URL Input & Preview: Enter any URL and view the rendered page directly in the application
  • Interactive Element Selection: Point-and-click interface for selecting target elements
  • Visual Data Preview: Real-time preview of extracted data
  • Drag-and-drop API Builder: Design your custom API endpoints with an intuitive interface
  • Live Documentation: Automatically generated API documentation based on your configuration
  • WebAssembly Integration: High-performance processing using Rust-compiled WebAssembly

🛠️ Tech Stack

  • Svelte
  • Embedded Headless Browser Integration
  • RESTful API Integration
  • Modern UI Component Library
  • TypeScript for Type Safety
  • Rust + WebAssembly for Performance-Critical Operations

📋 Prerequisites

  • Node.js (v16 or higher)
  • npm or yarn
  • Modern web browser
  • Backend API service running
  • Rust toolchain
  • wasm-pack

🚀 Running Locally

  1. Install wasm-pack:
cargo install wasm-pack
  1. Clone the repository (private access required)
git clone https://github.com/SrapeScribe/client.git
cd client
  1. Install dependencies:
npm install
  1. Build the WebAssembly:
npm run wasm:build
  1. Configure environment variables:
cp .env.example .env
  1. Start the development server:
npm run dev
  1. Navigate to http://localhost:5173/

🏗️ Project Structure

src/
├── components/
│   ├── BrowserView/        # Embedded browser component
│   ├── ElementSelector/    # Element selection tools
│   ├── DataPreview/       # Data preview panel
│   ├── ApiBuilder/        # API configuration interface
│   └── common/            # Shared UI components
├── services/
│   ├── api/              # Backend API integration
│   ├── browser/          # Browser control functions
│   └── selectors/        # Element selector logic
├── store/                # State management
├── types/                # TypeScript definitions
├── wasm/                 # WebAssembly modules
└── utils/                # Helper functions

🔍 Available Scripts

  • npm run dev - Start development server + build WebAssembly modules
  • npm run build - Build for production + build WebAssembly modules
  • npm run wasm:build - Build WebAssembly modules
  • npm run lint - Run linter
  • npm run format - Format code

📝 Development Guidelines

  • Follow the established code style and formatting rules
  • Update documentation as needed
  • Use semantic commit messages
  • Keep pull requests focused and manageable in size
  • Ensure crucial changes are approved by the team

📚 Documentation

Additional documentation is available in the docs/ directory:

📄 License

This software is proprietary. Copyright (c) 2024 MYKYTA VITKOVSKYI.

Academic Use

This software may be viewed and evaluated for educational purposes as part of a student project presentation.

Commercial and Production Use

All rights reserved. Unauthorized copying, modification, or distribution is prohibited. See the LICENSE file for details.

This is a student project with restricted usage rights. See LICENSE for details.