Page 1 of 1

POST and UTF-8

PostPosted: Sun May 28, 2017 3:45 am
by Lena
Hi.
I saved all *.php file in UTF-8.
What is the correct way to see Russian letters?

Code: Select all
//Windows 7
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
   std::unique_ptr<TStringList> paramlist(new TStringList());
   std::unique_ptr<TStringStream> Response(new TStringStream());
   String path = GetCurrentDir() + L"\\link.ini";
   std::unique_ptr<TIniFile> FileINI(new TIniFile(path));
   //http://mysite/pushTest/api.php
   String postlink = FileINI->ReadString(L"LINK", L"post", "");

   String postmetod = L"method=sendPush";
   String postheader = L"title=" + Trim(Edit1->Text);
   String postbody = L"text=" + Trim(Edit2->Text);
   paramlist->Add(postmetod);
   paramlist->Add(postheader);
   paramlist->Add(postbody);
   aHTTP->Post(postlink,paramlist.get(),Response.get());
   rezult = Response->DataString;
   ShowMessage(rezult);//I do not see Russian letters

Re: POST and UTF-8

PostPosted: Tue May 30, 2017 12:18 am
by Lena
It looks like this:
Code: Select all
std::unique_ptr<TStringStream> Response(new TStringStream("",TEncoding::UTF8, false));

Why after changing the encoding I can not parse JSON?
Code: Select all
String rezult = "";
   std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
   std::unique_ptr<TStringList> paramlist(new TStringList());
   std::unique_ptr<TStringStream> Response(new TStringStream("",TEncoding::UTF8, false));

   String path = GetCurrentDir() + L"\\link.ini";
   std::unique_ptr<TIniFile> FileINI(new TIniFile(path));
   //http://mysite/pushTest/api.php
   String postlink = FileINI->ReadString(L"LINK", L"post", "");

   String postmetod = L"method=sendPush1";
   String postheader = L"title=" + Trim(Edit1->Text);
   String postbody = L"text=" + Trim(Edit2->Text);
   paramlist->Add(postmetod);
   paramlist->Add(postheader);
   paramlist->Add(postbody);
   aHTTP->Post(postlink,paramlist.get(),Response.get());
   rezult = Response->DataString;

if(rezult != "")
   {
   ShowMessage(rezult);
   std::unique_ptr<TJSONValue> LJSONValue(TJSONObject::ParseJSONValue(rezult));
   TJSONObject *LJSONObject = dynamic_cast<TJSONObject*>(LJSONValue.get());
   if(LJSONObject != NULL)//problem always NULL
    {
          //
         }

Re: POST and UTF-8

PostPosted: Tue May 30, 2017 3:36 am
by Lena
Fix.
It is necessary to save file *.php in UTF-8 and without BOM!.

Re: POST and UTF-8

PostPosted: Thu Jun 01, 2017 5:21 pm
by rlebeau
Lena wrote:I saved all *.php file in UTF-8.


The encoding of the PHP itself is irrelevant. What is important is the encoding of the data it sends back to you.

Lena wrote:What is the correct way to see Russian letters?


Your code is reading the response into a TStringStream that is constructed without a TEncoding specified, so it will use TEncoding::Default, which is ANSI on Windows and UTF-8 on other platforms. If the response uses an encoding that does not match the stream's TEncoding object, the DataString property getter will run into conversion issues with non-ASCII characters. For that reason, I *NEVER* recommend reading an HTTP response into a TStringStream, because then you lose all control of charset handling.

You need to pay attention to the charset that is reported in the response and then decode the response data accordingly.

Had you used Indy''s TIdHTTP instead of Embarcadero's THTTPClient, it would have done the charset handling for you, eg:

Code: Select all
std::unique_ptr<TIdHTTP> aHTTP(new TIdHTTP());
...
String rezult = aHTTP->Post(postlink, paramlist.get());


With THTTPClient, try something more like this:

Code: Select all
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
...
String rezult = aHTTP->Post(postlink, paramlist.get())->ContentAsString();


Or:

Code: Select all
std::unique_ptr<THTTPClient> aHTTP(THTTPClient::Create());
...
_di_IHTTPResponse Response = aHTTP->Post(postlink, paramlist.get());
std::unique_ptr<TEncoding> Encoding(TEncoding::GetEncoding(Response->ContentCharSet));
String rezult = Response->ContentAsString(Encoding.get());


Lena wrote:It is necessary to save file *.php in UTF-8 and without BOM!.


Or, simply ignore the BOM if it is present:

Code: Select all
...
String rezult = ...
if (!result.IsEmpty())
{
    if (rezult[1] == 0xFEFF)
        result.Delete(1, 1);
    ...
}


Or, just let ParseJSONValue() handle the BOM for you:

Code: Select all
std::unique_ptr<TBytesStream> Response(new TBytesStream());
...
aHTTP->Post(postlink, paramlist.get(), Response.get());
if (Response->Size > 0)
{
    std::unique_ptr<TJSONValue> LJSONValue(TJSONObject::ParseJSONValue(Response->Bytes, 0, true));
    ...
}

Re: POST and UTF-8

PostPosted: Fri Jun 02, 2017 8:17 am
by Lena
Tnak You very much!