I want to retrieve content from web page. However, I tried above method but the error still come when the query string contain Chinese character.
code
$json = Get-Content -Encoding utf8 -Path "./kw.json" | ConvertFrom-Json
$kw = $json.name
$numPages = $json.page
# $encodingKW = [System.Text.Encoding]::UTF8.GetString([System.Text.Encoding]::GetEncoding("UTF-8").GetBytes($kw))
Write-Output $encodingKW
for ($i = 0; $i -lt $numPages; $i ++) {
$pn = $i * 50
[string]$url = "https://tieba.baidu.com/f?kw=${kw}&ie=utf-8&pn=${pn}"
# https://tieba.baidu.com/f?kw=%E5%88%80%E5%8A%8D%E7%A5%9E%E5%9F%9F&ie=utf-8&pn=0
Write-Output $url
$html = Invoke-WebRequest -Method Get -Uri $url
# $content = [System.Text.Encoding]::UTF8.GetString([System.Text.Encoding]::GetEncoding("iso-8859-1").GetBytes($html.Content))
Write-Output $html
break;
}
# [string]$url = "https://tieba.baidu.com/f?kw=${kw}"
# Write-Output $json.name
# Write-Output $json.page
# Write-Output $url
The url is
It give me Invoke-WebRequest error. I think the url is successfully encoding since the url is in encoding form ,backurl:'https://tieba.baidu.com/f?kw=%E5%9C%B0%E7%90%86&ie=utf-8&pn=0',.
But the error is
$html = Invoke-WebRequest -Method Get -Uri $url CategoryInfo : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest] WebException FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand
And Is there alternative method as python url = baseurl+"f?kw="+parse.quote(kw) + "&ie=utf-8&pn=" + str(i * 50)?
403(Forbidden)` error response.Invoke-WebRequestcmdlet has authentication-related parameters, driven by the-Authenticationparameter.