[weverse] add extractors #4768

bradenhilton · 2023-11-04T18:24:38Z

Closes #3995.

Weverse requires an account for paginated URLs. Its API is not publicly documented.

The API code is inspired by the yt-dlp implementation.

~~Speaking of yt-dlp, we could defer to it for media posts, as it supports VOD and live broadcasts as far as I can tell, but I haven't done this yet.~~

Some miscellaneous questions:

I noticed that you recently set Referer for requests by default, but it is absolutely mandatory for Weverse. Is it worth setting in the API headers?
The API responses use camelCase. Currently I am adding custom values to the metadata using snake_case. Should I stick to one style instead of mixing them?

bradenhilton · 2023-11-05T13:46:01Z

Example API responses:

Post (image)

{
  "extension": {},
  "body": "드디어 우리 노래 나왔잖아요!!\uD83D\uDC40\uD83E\uDEF6 \n피어나 어때요~~~??\n<w:attachment type=\"photo\" id=\"4-274880925\" />",
  "tags": [],
  "postId": "1-128752990",
  "postType": "NORMAL",
  "emotionCount": 30344,
  "commentCount": 3812,
  "publishedAt": 1698416087773,
  "membershipOnly": false,
  "availableActions": [
    "WRITE_COMMENT",
    "BOOKMARK",
    "WRITE_EMOTION",
    "READ_POST"
  ],
  "sectionType": "ARTIST",
  "hideFromArtist": false,
  "plainBody": "드디어 우리 노래 나왔잖아요!!\uD83D\uDC40\uD83E\uDEF6 \n피어나 어때요~~~??",
  "attachment": {
    "photo": {
      "4-274880925": {
        "url": "https://phinf.wevpstatic.net/MjAyMzEwMjdfMTMy/MDAxNjk4NDE2MDg3NDg0.uCJ26Yp7LLUBOJkCzex48s0BeG9Z8A4lgXZwWAqw64gg.c2KZ81NcEcFF4oJmGqpcuSH7MGAwwq9YkVIq3feLq04g.JPEG/image.jpg",
        "width": 2316,
        "height": 3088,
        "photoId": "4-274880925"
      }
    }
  },
  "shareUrl": "https://weverse.io/lesserafim/artist/1-128752990",
  "bookmarked": false,
  "viewerEmotionId": "3-1441778405",
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Post (video)

{
  "extension": {},
  "body": "너무 귀엽죠??\uD83E\uDEF6\n오늘부터 잘 부탁해요~~\uD83D\uDE09 <w:attachment type=\"video\" id=\"4-864785\" />",
  "attachment": {
    "video": {
      "4-864785": {
        "videoId": "4-864785",
        "uploadInfo": {
          "width": 720,
          "videoId": "FC3D5A7233E610269DD0BB16F34AC9175D92",
          "playTime": 3,
          "height": 720,
          "imageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMDVfMjU5/MDAxNjk2NDg5MTg0NTg3.MqPrC4culB2vLsB1wjdKLSgzHuIrwNyfbZ2NFVxIXZAg.q5HU37Askl9whC3wZ9lMOymXanLgm68UypqZRYKH8Kgg.JPEG/bf494c1d-634c-11ee-9671-80615f0bcd6e_01.jpg",
          "uploadInfoVersion": 1
        }
      }
    }
  },
  "emotionCount": 28853,
  "commentCount": 2379,
  "postId": "2-127465849",
  "tags": [],
  "publishedAt": 1696489184386,
  "sectionType": "ARTIST",
  "membershipOnly": false,
  "availableActions": [
    "BOOKMARK",
    "WRITE_COMMENT",
    "WRITE_EMOTION",
    "READ_POST"
  ],
  "hideFromArtist": false,
  "postType": "NORMAL",
  "plainBody": "너무 귀엽죠??\uD83E\uDEF6\n오늘부터 잘 부탁해요~~\uD83D\uDE09",
  "shareUrl": "https://weverse.io/lesserafim/artist/2-127465849",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Post (text)

{
  "extension": {},
  "tags": [],
  "publishedAt": 1696763700370,
  "emotionCount": 27997,
  "commentCount": 25772,
  "body": "피어나 뭐해요\uD83E\uDD79\uD83E\uDEF6",
  "availableActions": [
    "READ_POST",
    "BOOKMARK",
    "WRITE_COMMENT",
    "WRITE_EMOTION"
  ],
  "membershipOnly": false,
  "sectionType": "ARTIST",
  "plainBody": "피어나 뭐해요\uD83E\uDD79\uD83E\uDEF6",
  "postId": "3-135964325",
  "hideFromArtist": false,
  "attachment": {},
  "postType": "NORMAL",
  "shareUrl": "https://weverse.io/lesserafim/artist/3-135964325",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "artistReactions": [
    {
      "reactionType": "comment",
      "reactionCount": 19,
      "member": {
        "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
        "communityId": 47,
        "joined": true,
        "profileName": "쥬하\uD83C\uDF3F",
        "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
        "memberJoinStatus": "JOIN_COMMUNITY",
        "profileType": "ARTIST",
        "artistOfficialProfile": {
          "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
          "officialName": "KAZUHA",
          "birthday": {
            "date": 1060387200000,
            "dateString": "2003.08.09",
            "timezone": "Asia/Seoul",
            "bday": false
          }
        },
        "hasOfficialMark": true,
        "profileSpaceStatus": "ACCESSIBLE",
        "myProfile": false
      }
    }
  ],
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

Moment

{
  "extension": {
    "moment": {
      "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"940264D33932D0CD690A57AA820458275CD7\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"한국에서 처음으로 보낸 추석\uD83E\uDD0D\\n피어나 여러분 뭐해요~~??\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":0},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
      "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
      "expireAt": 1662784496362,
      "video": {
        "videoId": "4-83503",
        "uploadInfo": {
          "width": 1080,
          "videoId": "940264D33932D0CD690A57AA820458275CD7",
          "playTime": 10,
          "height": 1920,
          "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
          "uploadInfoVersion": 1
        }
      }
    }
  },
  "tags": [],
  "publishedAt": 1662698096362,
  "postType": "MOMENT",
  "sectionType": "MOMENT",
  "membershipOnly": false,
  "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
  "hideFromArtist": false,
  "emotionCount": 4490,
  "commentCount": 315,
  "availableActions": ["READ_POST"],
  "postId": "3-104040928",
  "attachment": {},
  "plainBody": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
  "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/3-104040928",
  "bookmarked": false,
  "locked": false,
  "hasProduct": false,
  "author": {
    "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
    "communityId": 47,
    "joined": true,
    "profileName": "쥬하\uD83C\uDF3F",
    "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
    "memberJoinStatus": "JOIN_COMMUNITY",
    "profileType": "ARTIST",
    "artistOfficialProfile": {
      "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
      "officialName": "KAZUHA",
      "birthday": {
        "date": 1060387200000,
        "dateString": "2003.08.09",
        "timezone": "Asia/Seoul",
        "bday": false
      }
    },
    "hasOfficialMark": true,
    "profileSpaceStatus": "ACCESSIBLE",
    "myProfile": false
  },
  "authorMomentPosts": {
    "paging": {},
    "data": [
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"940264D33932D0CD690A57AA820458275CD7\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"한국에서 처음으로 보낸 추석\uD83E\uDD0D\\n피어나 여러분 뭐해요~~??\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":0},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA5MDlfNTIg\\/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG\\/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
            "expireAt": 1662784496362,
            "video": {
              "videoId": "4-83503",
              "uploadInfo": {
                "width": 1080,
                "videoId": "940264D33932D0CD690A57AA820458275CD7",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1662698096362,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 4490,
        "commentCount": 315,
        "availableActions": ["READ_POST"],
        "postId": "3-104040928",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA5MDlfNTIg/MDAxNjYyNjk4MDg5MzI3.2G5v-YnreeHKRK0qU9T_5L0PtxKnjO-UxF5Iir4S2dQg.--uDITfIRkc2SJ5e9uDGB1udmfhRBoXVK13hhXcMT8Qg.JPEG/C467F406-294F-4D4B-891F-04FDB7F1E6AB.jpg",
              "playTime": 10,
              "cVideoId": "4-83503"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "한국에서 처음으로 보낸 추석\uD83E\uDD0D\n피어나 여러분 뭐해요~~??",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/3-104040928",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      },
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"45D6EC7EF41379A783252970FFAC04104648\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\\n\\n저녁에도 밝은 여름 최고\uD83E\uDD1F\\n\\n좋은 하루 보내세요~~!!\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":1},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MjVfNzgg\\/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG\\/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\n\n저녁에도 밝은 여름 최고\uD83E\uDD1F\n\n좋은 하루 보내세요~~!!",
            "expireAt": 1658825391456,
            "video": {
              "videoId": "3-15501",
              "uploadInfo": {
                "width": 1080,
                "videoId": "45D6EC7EF41379A783252970FFAC04104648",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA3MjVfNzgg/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1658738991456,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 6976,
        "commentCount": 468,
        "availableActions": ["READ_POST"],
        "postId": "4-101115099",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA3MjVfNzgg/MDAxNjU4NzM4OTg1MDUx.a5ffKA4rCn8iauE6VSqaC5o5QK0EoNTInA4KJKrcj-cg.S7nApdPG43lKfNGUEXA5kmPLQ-RN86D3J3u3mekW-u4g.JPEG/E48A02C9-A2FE-40E1-9C05-5AC5441A4EF7.jpg",
              "playTime": 10,
              "cVideoId": "3-15501"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "진짜 오늘 날씨 좋다~~~\uD83E\uDD0D\n저녁에도 밝은 여름 최고\uD83E\uDD1F\n좋은 하루 보내세요~~!!",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/4-101115099",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      },
      {
        "extension": {
          "moment": {
            "momentAsJson": "{\"description\":\"\",\"style\":\"\",\"version\":\"1.1.3\",\"clips\":[{\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\",\"events\":[],\"title\":\"\",\"videoId\":\"233B24713027EC3B674EDDE9FB61461CF404\",\"duration\":10,\"metadata\":{\"texts\":[{\"range\":{\"endTime\":10000,\"startTime\":0},\"text\":\"우와우\\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \\n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ\"}]},\"resolution\":{\"width\":1080,\"height\":1920}}],\"tags\":[],\"isReviewedByCustomer\":\"N\",\"title\":\"\",\"meta\":{\"stickerCnt\":{\"giphy\":3},\"mediaSrcCnt\":{\"video\":0,\"image\":1}},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\",\"duration\":10,\"indexes\":[{\"title\":\"\",\"range\":{\"endTime\":10000,\"startTime\":0},\"thumbnail\":\"https:\\/\\/weverse-phinf.pstatic.net\\/MjAyMjA3MTlfMjg1\\/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG\\/95CE3178-E10B-4298-B877-396268E6798F.jpg\"}],\"resolution\":{\"width\":1080,\"height\":1920}}",
            "body": "우와우\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ",
            "expireAt": 1658277011392,
            "video": {
              "videoId": "3-4450",
              "uploadInfo": {
                "width": 1080,
                "videoId": "233B24713027EC3B674EDDE9FB61461CF404",
                "playTime": 10,
                "height": 1920,
                "imageUrl": "https://phinf.wevpstatic.net/MjAyMjA3MTlfMjg1/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG/95CE3178-E10B-4298-B877-396268E6798F.jpg",
                "uploadInfoVersion": 1
              }
            }
          }
        },
        "tags": [],
        "publishedAt": 1658190611392,
        "postType": "MOMENT",
        "sectionType": "MOMENT",
        "membershipOnly": false,
        "hideFromArtist": false,
        "emotionCount": 4157,
        "commentCount": 252,
        "availableActions": ["READ_POST"],
        "postId": "4-100700103",
        "summary": {
          "videoCount": 1,
          "thumbnails": [
            {
              "type": "VIDEO",
              "url": "https://phinf.wevpstatic.net/MjAyMjA3MTlfMjg1/MDAxNjU4MTkwNjA1ODg3.qtMr3ltfmVDFt-XLFzXLLIjrJXu_On12_zsNPi6GYi4g.uAvNvpkpOJS6wwo1ckrIJeHSEM-shrNOMWW5WopHqXgg.JPEG/95CE3178-E10B-4298-B877-396268E6798F.jpg",
              "playTime": 10,
              "cVideoId": "3-4450"
            }
          ],
          "photoCount": 0,
          "snippets": []
        },
        "plainBody": "우와우\n이런 것 있었는지 몰랐어!!ㅋㅋㅋㅋ \n더 생기면 좋겠다\uD83E\uDEF6ㅋㅋㅋ",
        "shareUrl": "https://weverse.io/lesserafim/moment/b60d95bc3b71f4d97b28ac1b971cc641/post/4-100700103",
        "bookmarked": false,
        "locked": false,
        "hasProduct": false,
        "author": {
          "memberId": "b60d95bc3b71f4d97b28ac1b971cc641",
          "communityId": 47,
          "joined": true,
          "profileName": "쥬하\uD83C\uDF3F",
          "profileImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjQ2/MDAxNjk3NTU0OTg0MTcz.Ox-UsVCCMxdDfO5eTlOtS3rXjqoY0Rd1RxdN3rt3iTkg.PSneXamakKLjQuB8JR5Fs1R0WxUl5iXsGBS0V_6pzowg.JPEG/c88356d1-8f41-46ed-9f06-2d3810713011.jpeg",
          "memberJoinStatus": "JOIN_COMMUNITY",
          "profileType": "ARTIST",
          "artistOfficialProfile": {
            "officialImageUrl": "https://phinf.wevpstatic.net/MjAyMzEwMThfMjEz/MDAxNjk3NTU0OTc3NjAw.D9zosDQRuK28hOBJsF0JzIeXRn3rkSsXXKa2Japn4XEg.2dYdFv1-y1kjxEqXYtPunIzzGPl7syCSxcbN-Ni0mT4g.JPEG/6faffa29-8c53-4211-b025-69ca813c9ad3.jpeg",
            "officialName": "KAZUHA",
            "birthday": {
              "date": 1060387200000,
              "dateString": "2003.08.09",
              "timezone": "Asia/Seoul",
              "bday": false
            }
          },
          "hasOfficialMark": true,
          "profileSpaceStatus": "ACCESSIBLE",
          "myProfile": false
        },
        "community": {
          "communityId": 47,
          "communityName": "LE SSERAFIM",
          "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
          "artistCode": "LESSERAFIM"
        }
      }
    ]
  },
  "community": {
    "communityId": 47,
    "communityName": "LE SSERAFIM",
    "logoImage": "https://phinf.wevpstatic.net/MjAyMzEwMTJfMTI4/MDAxNjk3MDc5MTA3Njg4.tWmeZYTw5wXcgDYW1CdqRI9Sn81B7Fd2V2CF7Qt7bXkg.hOVkhjZpWx8q4YS0LHinzGDsaptQFkM1CvgKEu9aLJgg.PNG/6a41660d-68a1-40a7-8e90-0687914cbc77.png",
    "artistCode": "LESSERAFIM",
    "homeGradationColor": {
      "upLeftColorCode": "#7887c3",
      "upRightColorCode": "#455698",
      "downLeftColorCode": "#c0cbf5",
      "downRightColorCode": "#8e9edf",
      "menuTabLeftColorCode": "#465388",
      "menuTabRightColorCode": "#1b2e79"
    }
  }
}

For endpoints with pagination support, their response includes a paging.nextParams.after value that can be sent as an after param in the next request.

Some of these values could probably be removed from the final metadata (or the metadata could be created from scratch with cherry picked values), though I'm hesitant to remove too much for versatility with naming/filtering.

bradenhilton · 2023-11-05T22:56:50Z

I found myself needing to pull an all-nighter, so I went ahead and just implemented most of the TODOs to keep myself busy.

Some of the extractors probably aren't necessary. WeverseMomentsExtractor in particular could probably be merged into WeverseMomentExtractor, and an optional capture group could be used for single Moments instead. I wanted to keep things separate for now as I'm still learning how everything works.

bradenhilton · 2023-11-12T01:07:39Z

I found a new endpoint which was the missing piece to allow all non-paginating extractors to work without authentication. I have now added some more robust tests.

bradenhilton · 2024-02-11T19:41:50Z

@mikf I believe this is pretty much finished now (pending a review at your convenience, of course).

I attempted to rewrite it in the style of some other extractors such as twitter and bluesky (e.g. using something like posts() instead of items() where possible), but I'm unsure if there are enough similarities between Weverse and those sites for it to be of any use, so I reverted to delegating for all extractors that use pagination.

ytdl support (for non-embed videos) has been removed. Only yt-dlp has Weverse extractors (for Media videos and live streams/VODs, not Post videos), I am not sure if ytdl support adds much value here. ytdl is still used if embeds is enabled.

I also have most if not all possible relevant API responses saved if you'd like them for context (or to create the extractors yourself if you prefer).

Hrxn · 2024-05-31T14:27:38Z

@mikf

TakeoIschiFan · 2024-08-17T08:35:17Z

+1
would like to see this merged

mattbarham · 2024-08-21T14:22:57Z

+1 pleaseee

Sylphystia · 2024-11-26T09:43:37Z

Any news on this being implemented?

bradenhilton · 2024-11-26T10:21:54Z

@Sylphystia This PR is outdated and I haven't gotten the time to update it yet.

For now you can find it in my custom modules repo.

bradenhilton force-pushed the weverse branch from 4b512b6 to 3b60dab Compare November 4, 2023 18:40

bradenhilton force-pushed the weverse branch from 3b60dab to 3dfc3ce Compare November 5, 2023 22:55

bradenhilton force-pushed the weverse branch 2 times, most recently from cbb6bc7 to 6f16e1a Compare November 7, 2023 10:13

bradenhilton marked this pull request as ready for review November 7, 2023 10:14

This comment was marked as outdated.

Sign in to view

bradenhilton force-pushed the weverse branch 3 times, most recently from 55c8167 to 7954d84 Compare November 12, 2023 01:02

bradenhilton force-pushed the weverse branch from 7954d84 to fff03a4 Compare November 13, 2023 01:47

bradenhilton force-pushed the weverse branch 6 times, most recently from a2ad565 to 963d311 Compare February 11, 2024 02:55

bradenhilton force-pushed the weverse branch from 963d311 to ff661fa Compare May 31, 2024 12:49

bradenhilton force-pushed the weverse branch from ff661fa to d526bf6 Compare August 18, 2024 15:03

[weverse] add extractors

b1b1cfe

bradenhilton force-pushed the weverse branch from d526bf6 to b1b1cfe Compare August 18, 2024 15:06

bradenhilton marked this pull request as draft November 26, 2024 10:22

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[weverse] add extractors #4768

[weverse] add extractors #4768

bradenhilton commented Nov 4, 2023 •

edited

Loading

bradenhilton commented Nov 5, 2023

bradenhilton commented Nov 5, 2023

This comment was marked as outdated.

bradenhilton commented Nov 12, 2023

bradenhilton commented Feb 11, 2024 •

edited

Loading

Hrxn commented May 31, 2024

TakeoIschiFan commented Aug 17, 2024

mattbarham commented Aug 21, 2024

Sylphystia commented Nov 26, 2024

bradenhilton commented Nov 26, 2024

[weverse] add extractors #4768

Are you sure you want to change the base?

[weverse] add extractors #4768

Conversation

bradenhilton commented Nov 4, 2023 • edited Loading

bradenhilton commented Nov 5, 2023

bradenhilton commented Nov 5, 2023

This comment was marked as outdated.

bradenhilton commented Nov 12, 2023

bradenhilton commented Feb 11, 2024 • edited Loading

Hrxn commented May 31, 2024

TakeoIschiFan commented Aug 17, 2024

mattbarham commented Aug 21, 2024

Sylphystia commented Nov 26, 2024

bradenhilton commented Nov 26, 2024

bradenhilton commented Nov 4, 2023 •

edited

Loading

bradenhilton commented Feb 11, 2024 •

edited

Loading