Uploaded image for project: 'Spring Roo'
  1. Spring Roo
  2. ROO-1684

The UTF8 (german and cyrillic) data is not correctly written to the database

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Complete
    • Affects Version/s: 1.1.0.RELEASE
    • Fix Version/s: 1.1.1.RELEASE
    • Component/s: WEB MVC
    • Labels:
      None
    • Environment:
      Windows XP, tomcat

      Description

      The utf8 characters are not correctly written to database.

      a) german special characters (ö,ä,ß,ü)
      b) cyrillic characters (щвды...)

      See
      http://forum.springsource.org/showthread.php?p=327431#post327431
      and
      https://jira.springsource.org/browse/ROO-89?focusedCommentId=60102#action_60102

      I did some tests already, it didnt help:

      1. Set the CharacterEncodingFilter filter in web.xml to first place
      2. Set <property name="hibernate.connection.charSet" value="UTF-8" /> in persistence xml
      3. I updated the DB manually with a correct UTF8 data (copied from hsqldb from same application but in grails) and is showed correctly, so the utf8 data is correcly read\displayed from database.

      I have same application in grails. Same db (hsqldb), db settings, db version.

      Here the sample of data in:

      Grails:
      Entered value "Привет München",
      value in db "INSERT INTO GUETER_EINHEIT VALUES(17,0,'\u041f\u0440\u0438\u0432\u0435\u0442 M\u00fcnchen')"

      Roo:
      Entered value "привет münchen",
      value in the db "INSERT INTO GUETER_EINHEIT VALUES(2,'\u00d0\u009f\u00d1\u0080\u00d0\u00b8\u00d0\u00b2\ u00d0\u00b5\u00d1\u0082 m\u00c3\u00bcnchen',0)"

      Very strange, I see in Roo for one special character two unicode characters and the generated unicode of grails and roo is different

        Issue Links

          Activity

          Hide
          berto Ro added a comment -

          Hi,
          same problem in my case. None of the workarounds/fixes suggested above was helpful.

          Setting a breakpoint in *Controller.update() method and modifying manually the problematic value to "proper" one just before calling the *.merge() causes the string to be updated to the DB (and then retrieved and updated on the view) correctly.

          Roo 1.1.0.RELEASE
          STS 2.5.0.RELEASE
          MySQL 5.1.51 community

          Berto

          Show
          berto Ro added a comment - Hi, same problem in my case. None of the workarounds/fixes suggested above was helpful. Setting a breakpoint in *Controller.update() method and modifying manually the problematic value to "proper" one just before calling the *.merge() causes the string to be updated to the DB (and then retrieved and updated on the view) correctly. Roo 1.1.0.RELEASE STS 2.5.0.RELEASE MySQL 5.1.51 community Berto
          Hide
          atsuk Ats U. added a comment -

          The description of this issue looks very similar to what I had after updating to Roo 1.1.0.RC1. I solved my problem manually by reordering filter mappings in web.xml as I described when I created ROO-1731 - it might be worth looking at.

          Reordering filters were mentioned here, but note that the order of filter elements is not important - the order of filter-mapping elements are that matter. Maybe that is why some of You didn't get expected results?
          Also I didn't put CharacterEncodingFilter first (just first of those three filter mappings I concidered importaint in the context of ROO-1731) - I had Spring OpenEntityManagerInViewFilter befor those i mentioned in ROO-1731.

          Stefan, if your solution is what I described with ROO-1731, then You could probably close that issue

          Ats U.

          Show
          atsuk Ats U. added a comment - The description of this issue looks very similar to what I had after updating to Roo 1.1.0.RC1. I solved my problem manually by reordering filter mappings in web.xml as I described when I created ROO-1731 - it might be worth looking at. Reordering filters were mentioned here, but note that the order of filter elements is not important - the order of filter-mapping elements are that matter. Maybe that is why some of You didn't get expected results? Also I didn't put CharacterEncodingFilter first (just first of those three filter mappings I concidered importaint in the context of ROO-1731 ) - I had Spring OpenEntityManagerInViewFilter befor those i mentioned in ROO-1731 . Stefan, if your solution is what I described with ROO-1731 , then You could probably close that issue Ats U.
          Hide
          sschmidt Stefan Schmidt added a comment -

          With commit 2f373ffa30c5ceeccd53a9d21e64f6c0006d1d6e the following changes were made:

          Filter mappings are now installed in the following sequence:

          1. CharacterEncodingFilter
          2. HttpMethodFilter
          (3. springSecurityFilterChain)
          4. OpenEntityManagerInViewFilter

          default.jspx contains a pageEncoding page directive: <jsp:directive.page pageEncoding="UTF-8" />

          persistence.xml now contains: <property name="hibernate.connection.charSet" value="UTF-8"/> if Hibernate is selected.

          It would be great if I can get some feedback so we can close this ticket.

          Show
          sschmidt Stefan Schmidt added a comment - With commit 2f373ffa30c5ceeccd53a9d21e64f6c0006d1d6e the following changes were made: Filter mappings are now installed in the following sequence: 1. CharacterEncodingFilter 2. HttpMethodFilter (3. springSecurityFilterChain) 4. OpenEntityManagerInViewFilter default.jspx contains a pageEncoding page directive: <jsp:directive.page pageEncoding="UTF-8" /> persistence.xml now contains: <property name="hibernate.connection.charSet" value="UTF-8"/> if Hibernate is selected. It would be great if I can get some feedback so we can close this ticket.
          Hide
          nobody__ Fokko Breden added a comment -

          Stefan, changing filter mapping works. We have moved

          <filter-mapping>
          <filter-name>CharacterEncodingFilter</filter-name>
          <url-pattern>/*</url-pattern>
          </filter-mapping>

          to the first position of all <filter-mapping> entries. Posting UTF-8 is now OK.

          (see also: https://jira.springsource.org/browse/ROO-1731)

          Show
          nobody__ Fokko Breden added a comment - Stefan, changing filter mapping works. We have moved <filter-mapping> <filter-name>CharacterEncodingFilter</filter-name> <url-pattern>/*</url-pattern> </filter-mapping> to the first position of all <filter-mapping> entries. Posting UTF-8 is now OK. (see also: https://jira.springsource.org/browse/ROO-1731 )
          Hide
          sschmidt Stefan Schmidt added a comment -

          I'll go ahead and mark this as resolved.

          Show
          sschmidt Stefan Schmidt added a comment - I'll go ahead and mark this as resolved.

            People

            • Assignee:
              sschmidt Stefan Schmidt
              Reporter:
              eugeis Eugen Eisler
            • Votes:
              5 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: