爬虫之Boss直聘 招聘数据采集（zp

前言

大家早好、午好、晚好吖 ❤ ~欢迎光临本文章

如果有什么疑惑/资料需要的可以点击文章末尾名片领取源码

课程亮点:

1、动态数据抓包

2、JSON数据解析方法

环境介绍:

• python 3.8 解释器: 执行代码的

• pycharm 专业版: 写代码的 帮助我们 快速开发代码的

• requests >>> pip install requests

• sekiro >>> RPC框架

  抓包分析

  确定需要的数据 然后找该数据的来源

  https://www.zhipin.com/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page=1&pageSize=30

  代码实现

  1. 发送请求

  2. 获取数据

  3. 解析数据

  代码展示

  '''
  python资料获取看这里噢！！ 小编 V：qian97378，即可获取：
  文章源码/教程/资料/解答等福利，还有不错的视频学习教程和PDF电子书！
  '''
  import requests
  import csv
  with open('boss.csv', mode='w', newline='', encoding='utf-8') as f:
      csv.writer(f).writerow(['jobName', 'cityName', 'areaDistrict', 'businessDistrict', 'salaryDesc', 'jobExperience', 'jobDegree', 'skills', 'brandName', 'brandIndustry', 'brandStageName', 'brandScaleName', 'welfareList'])
  # 请求头
  headers = { 'Referer': 'https://www.zhipin.com/web/geek/job?query=python%E7%88%AC%E8%99%AB&city=100010000&page=1',
      #          wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0Skx2DUZ5fiBSb0RZDWpRc3ciERc6aTt3IB5UeRZzNyVTO2s9VRI5D24SQDxkDzVdfi9bSVQWXQZtQUcefiNsFCk5FHMuZHlKCVVkOnIELDdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.com%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75
      'Cookie': 'wd_guid=74818318-ca4c-4c55-8023-515796a1af9a; historyState=state; _bl_uid=gLl3kn8OimXutaow0kgvk3sq8qbh; __g=-; Hm_lvt_194df3105ad7148dcf2b98a91b5e727a=1699018011,1699707457,1699962314,1699965523; Hm_lpvt_194df3105ad7148dcf2b98a91b5e727a=1699965558; __zp_stoken__=3f6ceWFc7eFF0SnAKXyRmfiBSb0Q1dz8TTzUiERc6aRxqORo%2BeRZzNyVTeiwdMmkRD24SQDxkDzVzfntbRwlYVU9KZxESMRhUGQ0DFHMuZHlKCX0fXVJDbTdGfhZeASBxTVcgPzgbDT9tbCE%3D; __c=1699963318; __l=l=%2Fwww.zhipin.com%2Fweb%2Fgeek%2Fjob%3Fquery%3Dpython%25E7%2588%25AC%25E8%2599%25AB%26city%3D100010000%26page%3D1&r=&g=&s=3&friend_source=0&s=3&friend_source=0; __a=40359856.1696853365.1699962313.1699963318.75.11.11.75',
      'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/119.0.0.0 Safari/537.36'
  }
  for page in range(1, 31):
      url = f'https://www.zhipin.com/wapi/zpgeek/search/joblist.json?scene=1&query=python%E7%88%AC%E8%99%AB&city=100010000&experience=&payType=&partTime=°ree=&industry=&scale=&stage=&position=&jobType=&salary=&multiBusinessDistrict=&multiSubway=&page={page}&pageSize=30'
      # 1. 发送请求
      response = requests.get(url=url, headers=headers)
      # 2. 获取数据
      json_data = response.json()
      # 3. 提取数据
      jobList = json_data['zpData']['jobList']
      for job in jobList:
          jobName = job['jobName']
          cityName = job['cityName']
          areaDistrict = job['areaDistrict']
          businessDistrict = job['businessDistrict']
          salaryDesc = job['salaryDesc']
          jobExperience = job['jobExperience']
          jobDegree = job['jobDegree']
          skills = ' '.join(job['skills'])
          brandName = job['brandName']
          brandIndustry = job['brandIndustry']
          brandStageName = job['brandStageName']
          brandScaleName = job['brandScaleName']
          welfareList = ' '.join(job['welfareList'])
          print(jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList)
          with open('boss.csv', mode='a', newline='', encoding='utf-8') as f:
              csv.writer(f).writerow([jobName, cityName, areaDistrict, businessDistrict, salaryDesc, jobExperience, jobDegree, skills, brandName, brandIndustry, brandStageName, brandScaleName, welfareList])
  

  尾语

  好了，今天的分享就差不多到这里了！

  对下一篇大家想看什么，可在评论区留言哦！看到我会更新哒(ง •_•)ง

  喜欢就关注一下博主，或点赞收藏评论一下我的文章叭！！！

  最后，宣传一下呀~👇👇👇 更多源码、资料、素材、解答、交流 皆点击下方名片获取呀👇👇👇