Logo
Linkedin Api

LinkedIn Profile API: How to Get Full LinkedIn Profiles

Learn how to retrieve comprehensive LinkedIn profile data using Python with BeautifulSoup and the more powerful Rapid API approach, without complex OAuth authentication.

LinkedIn Profile API: How to Get Full LinkedIn Profiles

LinkedIn Profile API: How to Get Full LinkedIn Profiles

In this comprehensive guide, we'll explore:

  • Why LinkedIn profile data is valuable for businesses and developers
  • Using Python with BeautifulSoup for basic LinkedIn scraping
  • The preferred method: Rapid API's LinkedIn Profile APIs
  • Ethical considerations and best practices

The Value of LinkedIn Profile Data

LinkedIn hosts over 900 million professional profiles across more than 200 countries, making it the world's largest professional database. Access to this data can provide tremendous value for:

  • Recruiters: Finding qualified candidates and verifying work history
  • Sales teams: Identifying decision-makers and building targeted prospect lists
  • Marketers: Creating persona-based campaigns with accurate professional data
  • Researchers: Analyzing professional networks and industry trends

While LinkedIn offers an official API, it's heavily restricted and requires complex OAuth authentication with limited endpoints. For many use cases, alternative methods are necessary.

Method 1: Python Scraping with BeautifulSoup

Let's start with a basic Python approach using BeautifulSoup. This method has limitations but can help you understand the fundamentals.

Setting Up Your Environment

# Install required packages
pip install requests beautifulsoup4 lxml

# Import libraries
import requests
from bs4 import BeautifulSoup
import time
import random
import json

Basic Profile Scraping Function

def scrape_linkedin_profile(profile_url):
    """
    Scrape basic information from a LinkedIn profile

    Parameters:
        profile_url (str): The URL of the LinkedIn profile

    Returns:
        dict: Profile information (name, headline, about, etc.)
    """
    # Headers to mimic a browser visit
    headers = {
        'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/96.0.4664.110 Safari/537.36',
        'Accept-Language': 'en-US,en;q=0.9',
    }

    try:
        # Send request
        response = requests.get(profile_url, headers=headers)
        response.raise_for_status()  # Raise an exception for HTTP errors

        # Parse HTML
        soup = BeautifulSoup(response.text, 'lxml')

        # Extract basic information
        profile_data = {}

        # Name
        name_tag = soup.find('h1', {'class': 'text-heading-xlarge'})
        profile_data['name'] = name_tag.get_text().strip() if name_tag else None

        # Headline
        headline_tag = soup.find('div', {'class': 'text-body-medium'})
        profile_data['headline'] = headline_tag.get_text().strip() if headline_tag else None

        # About
        about_section = soup.find('div', {'id': 'about'})
        if about_section:
            about_tag = about_section.find('div', {'class': 'display-flex'})
            profile_data['about'] = about_tag.get_text().strip() if about_tag else None

        # Location
        location_tag = soup.find('span', {'class': 'text-body-small'})
        profile_data['location'] = location_tag.get_text().strip() if location_tag else None

        return profile_data

    except Exception as e:
        print(f"Error scraping profile: {e}")
        return None

Challenges with Direct Scraping

While the above approach might work for basic public information, it faces several significant challenges:

  1. LinkedIn's Anti-Scraping Measures: LinkedIn employs sophisticated techniques to detect and block scrapers.
  2. Frequent Layout Changes: LinkedIn regularly updates its HTML structure, breaking scraping code.
  3. Login Requirements: Most valuable profile data is behind login walls.
  4. Rate Limiting: LinkedIn restricts the number of profiles you can view in a short time.
  5. Legal Concerns: LinkedIn's Terms of Service prohibit scraping, potentially exposing you to legal issues.

This is why a more reliable solution is needed for production use.

Method 2: The Preferred Approach - Rapid API for LinkedIn Profiles

Using a specialized API service like those available on Rapid API provides a more robust, ethical, and reliable solution.

Why Choose Rapid API for LinkedIn Profile Data?

  • No OAuth Required: Simplified authentication with API keys
  • Comprehensive Data: Access complete profile information
  • Maintained Endpoints: APIs are regularly updated to adapt to LinkedIn changes
  • Legal Compliance: Providers ensure their methods comply with applicable regulations
  • Reliable Performance: Stable endpoints with high uptime guarantees

Step 1: Set Up Your Rapid API Account

  1. Visit Rapid API and create an account
  2. Redirect to the Fresh LinkedIn Scraper API
  3. Subscribe to a service like the "Fresh LinkedIn Scraper API"
  4. Obtain your API key from the dashboard

Step 2: Retrieve LinkedIn Profile Data with Python

Here's how to fetch a complete LinkedIn profile using Python and the Fresh LinkedIn Scraper API:

import requests
import json

def get_linkedin_profile(username):
    """
    Retrieve detailed LinkedIn profile data using Rapid API

    Parameters:
        username (str): LinkedIn username (e.g., 'williamhgates')

    Returns:
        dict: Complete profile information
    """
    url = "https://fresh-linkedin-scraper-api.p.rapidapi.com/api/v1/user/profile"

    querystring = {"username": username}

    headers = {
        "x-rapidapi-key": "YOUR_RAPID_API_KEY",  # Replace with your actual API key
        "x-rapidapi-host": "fresh-linkedin-scraper-api.p.rapidapi.com"
    }

    try:
        response = requests.get(url, headers=headers, params=querystring)
        response.raise_for_status()

        # Parse JSON response
        profile_data = response.json()

        # Check if request was successful
        if profile_data.get("success"):
            return profile_data.get("data")
        else:
            print(f"API Error: {profile_data.get('message')}")
            return None

    except Exception as e:
        print(f"Error retrieving profile: {e}")
        return None

# Example usage
profile = get_linkedin_profile("williamhgates")
print(json.dumps(profile, indent=2))

Example API Response

The API returns comprehensive profile data in a structured JSON format:

{
  "id": "251749025",
  "urn": "ACoAAA8BYqEBCGLg_vT_ca6mMEqkpp9nVffJ3hc",
  "public_identifier": "williamhgates",
  "first_name": "Bill",
  "last_name": "Gates",
  "full_name": "Bill Gates",
  "headline": "Chair, Gates Foundation and Founder, Breakthrough Energy",
  "location": {
    "country": "United States",
    "country_code": "US",
    "city": "Seattle, Washington, United States"
  },
  "background": {
    "education": [
      {
        "school_name": "Harvard University",
        "school_linkedin_profile_url": "https://www.linkedin.com/school/harvard-university/",
        "degree_name": "Drop Out",
        "field_of_study": "",
        "starts_at": {
          "day": 1,
          "month": 9,
          "year": 1973
        },
        "ends_at": {
          "day": 1,
          "month": 3,
          "year": 1975
        }
      }
    ],
    "experience": [
      {
        "company": "Gates Foundation",
        "company_linkedin_profile_url": "https://www.linkedin.com/company/bill-&-melinda-gates-foundation/",
        "title": "Co-chair",
        "description": "",
        "location": "Seattle, WA",
        "starts_at": {
          "day": 1,
          "month": 1,
          "year": 2000
        },
        "ends_at": null,
        "company_logo_url": "https://media.licdn.com/dms/image/C4E0BAQEzlqXUZUZNnw/company-logo_100_100/0/1656681489088?e=1747286400&v=beta&t=A-..."
      },
      {
        "company": "Microsoft",
        "company_linkedin_profile_url": "https://www.linkedin.com/company/microsoft/",
        "title": "Co-founder",
        "description": "",
        "location": "Redmond, WA",
        "starts_at": {
          "day": 4,
          "month": 4,
          "year": 1975
        },
        "ends_at": null,
        "company_logo_url": "https://media.licdn.com/dms/image/C560BAQE88xCsONDULQ/company-logo_100_100/0/1630430863501?e=1747286400&v=beta&t=..."
      }
    ]
  },
  "skills": [
    "Entrepreneurship",
    "Leadership",
    "Philanthropy",
    "Business Strategy",
    "Software Development"
  ],
  "follower_count": 35486921,
  "is_premium": true,
  "is_influencer": true,
  "avatar": [
    {
      "width": 800,
      "height": 800,
      "url": "https://media.licdn.com/dms/image/v2/D5603AQF-RYZP55jmXA/profile-displayphoto-shrink_800_800/..."
    }
  ]
}

Step 3: Additional Endpoints Available

The LinkedIn Profile API on Rapid API offers several specialized endpoints for different data needs:

  • /user/profile: Get core profile data (as shown above)
  • /user/skills: Retrieve full skills list
  • /user/experience: Get detailed work history
  • /user/education: Access education background ... etc

Step 4: Practical Applications

Let's explore a real-world example of using this API for lead enrichment:

import pandas as pd
import requests
import time

def enrich_leads_with_linkedin_data(leads_csv_path, output_csv_path):
    """
    Enrich a list of leads with LinkedIn profile data

    Parameters:
        leads_csv_path (str): Path to CSV with lead data (must include linkedin_username column)
        output_csv_path (str): Path to save enriched data
    """
    # Load leads data
    leads_df = pd.read_csv(leads_csv_path)

    # Check if linkedin_username column exists
    if 'linkedin_username' not in leads_df.columns:
        print("Error: CSV must contain 'linkedin_username' column")
        return

    # API configuration
    url = "https://fresh-linkedin-scraper-api.p.rapidapi.com/api/v1/user/profile"
    headers = {
        "x-rapidapi-key": "YOUR_RAPID_API_KEY",
        "x-rapidapi-host": "fresh-linkedin-scraper-api.p.rapidapi.com"
    }

    # Enrichment columns to add
    new_columns = [
        'linkedin_headline', 'linkedin_location', 'linkedin_company',
        'linkedin_title', 'linkedin_skills'
    ]

    # Add new columns to DataFrame
    for column in new_columns:
        leads_df[column] = None

    # Process each lead
    for index, row in leads_df.iterrows():
        username = row['linkedin_username']

        if pd.isna(username) or username == '':
            continue

        print(f"Processing lead {index+1}/{len(leads_df)}: {username}")

        try:
            # Call API
            response = requests.get(
                url,
                headers=headers,
                params={"username": username}
            )

            if response.status_code == 200:
                data = response.json().get("data", {})

                # Update DataFrame with LinkedIn data
                leads_df.at[index, 'linkedin_headline'] = data.get('headline', '')

                # Location
                location = data.get('location', {})
                location_str = f"{location.get('city', '')}"
                leads_df.at[index, 'linkedin_location'] = location_str

                # Current position
                experiences = data.get('background', {}).get('experience', [])
                if experiences and len(experiences) > 0:
                    current_job = experiences[0]  # Most recent job
                    leads_df.at[index, 'linkedin_company'] = current_job.get('company', '')
                    leads_df.at[index, 'linkedin_title'] = current_job.get('title', '')

                # Skills
                skills = data.get('skills', [])
                leads_df.at[index, 'linkedin_skills'] = ', '.join(skills) if skills else ''

            else:
                print(f"  Error: HTTP {response.status_code}")

            # Respect rate limits
            time.sleep(1)

        except Exception as e:
            print(f"  Error processing {username}: {e}")

    # Save enriched data
    leads_df.to_csv(output_csv_path, index=False)
    print(f"Enriched data saved to {output_csv_path}")

# Example usage
enrich_leads_with_linkedin_data('leads.csv', 'enriched_leads.csv')

Pricing and API Plans

The Rapid API marketplace offers various LinkedIn Profile API options with different pricing tiers:

  • Free Tier: Limited requests per month (typically 100-500)
  • Basic Plan: Starting around $29/month for 1,000+ requests
  • Professional Plan: $99/month for 5,000+ requests with priority support
  • Enterprise Plan: Custom pricing for high-volume needs

Visit the Rapid API Marketplace to compare different LinkedIn API providers and their pricing models.

Ethical Considerations and Best Practices

When working with LinkedIn profile data, follow these guidelines:

  1. Respect Privacy: Only collect data you genuinely need for your use case
  2. Transparent Usage: Be clear about how you're using the data
  3. Data Security: Store any collected information securely
  4. Rate Limiting: Follow the API provider's rate limits
  5. Terms Compliance: Ensure your use case complies with both LinkedIn's and the API provider's terms

Conclusion

Accessing LinkedIn profile data programmmatically provides valuable insights for business intelligence, recruitment, sales, and marketing. While direct scraping with Python and BeautifulSoup offers a basic approach, it faces significant challenges and potential legal issues.

The preferred method is using specialized LinkedIn Profile APIs through marketplaces like Rapid API. These services provide reliable, compliant access to comprehensive profile data without the complexity of OAuth authentication.

Ready to get started? Visit the Rapid API Marketplace to find the right LinkedIn Profile API for your needs. Choose a provider that offers the specific endpoints you require and a pricing tier that matches your volume requirements.